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Highlights 



Evidence from boih grantee-provided mail questionnaires and in-person visits lo grantee institution^ 
indicates thai CSIP projects are affording students previously unavailable opportunities lor direct, hand- 
on learning that significant!) siimulaie. enrich and enliven their undergraduate experiences. Man> 
instances were found where CSIP-supported projects are thought lo have produced increased 
department enrollments, increased student interest in science/technology careers, and increased posi- 
college employment opportunities for students in science-related occupations. 

CSIP projects have consistently had substantial positive impacts on faculty and departments, both h\ 
providing a prestigious form of recognition of merit and - more importantly - by encouraging and 
enabling faculty to pursue their ideas for updating and improving undergraduate curricula. As well as 
often having beneficial effects on individual faculty member's careers, many CSIP projects have palpably 
reinvigorated faculty morale and enthusiasm for leaching. 

From the initial impetus provided by CSIP grants, many projects have attracted additional financial and 
other support enabling them lo obtain additional instructional equipment and to grow in scope well 
beyond what had initially been contemplated. On average, CSIP projects attracted total financial 
support at least 4.5 times the size of the NSF grant amount. This often resulted in the allocation ol 
additional laboratory space for undergraduate instruction, and in some cases led to the creation ol new 
tenure-track faculty positions and new undergraduate degree programs. 

Site visits to a representative sample of funded CSIP projects found: 

Most have generated educational and other impacts far in excess of what might have been 
expected from the modest financial investment represented bv ihe CSIP grant; and 

Most would not have been implemented, or would have been significantly scaled back, if the 
grant had not been received. 

The CSIP program appears to have had beneficial effects even among unsuccessful grant applicants, 
some of whom later went on to find other sources of financial support for the curriculum plans the\ 
developed in the course of the CSIP application process. However, most CSIP applicants who did not 
receive NSF grant support for their projects were not able to implement their ideas as fully as they had 
proposed, and more than a third have been unable to obtain any of the instrumentation their projects 
required. 

Women, minorities, and the handicapped were well represented among CSIP grantees, at levels above 
their representation in the current national population of scientists and engineers. Responsiveness to 
CSIP program objectives and persistence in submitting revised or new proposals if not initially funded 
were the only other proposal, investigator, department, or institution characteristics found statistically 
associated with success in winning CSIP grants. Grants appear to have been widely and equitably 
distributed across institutions and across science/engineering disciplines, in accord with program 
objectives. 



3 



ASSESSMENT OF THE 
NATIONAL SCIENCE 
FOUNDATION'S 1985-87 

v 

COLLEGE SCIENCE 
INSTRUMENTATION PROGRAM 



Kenneth Burgdorf 
Carin A. Celebuski 
Westat. Inc. 



WESTAT 



An Employee Owned Research Corporation OCTOBER 1 990 



ERIC 



4 



The National Science Foundation has TDD (Telephonic Device for the Deaf) capability, 
which enables individuals with hearing impairments to communicate with the Division 
of Personnel Management about NSF programs, employment, or general information. 
This number is (202) 357-7492. 



5 



ERIC 



PREFACE 



ERIC 



This report summarizes the findings and conclusions of an assessment of the first three years of the National 
Science Foundation's College Science Instrumentation Program (CSIP). in 1985-87, the period covered bs this 
evaluation, CSIP avarded grants on a competitive basis to departments at a total of 410 non-doctorale-granling 
colleges and universities for the implementation of institution-based projects to improve undergraduate 
science/engineering instruction. Grants awarded under CSIP provided matching funds of up to $50,000 for the 
purchase of laboratory instrumentation for use in such projects. 

In 1988, NSF awarded a contract to Wcslal, Inc. to conduct an independent assessment of CSIP during its first 
three years of operation and to develop recommendations to the Foundation regarding future modifications of 
the program. This intensive two-year study involved site visits to institutions with CSIP grants and statistical 
analyses of mailed questionnaire data from both successful and unsuccessful CSIP applicants. The extensive 
data assembled in the course of this study are summarized in this report. 

As part of the assessment, an Advisory Panel was appointed to oversee the development of evaluation 
instruments and to ensure both the integrity of the review process and the validity of the study findings. An 
interpretive overview of the CSIP program and the results of the assessment has been prepared by the members 
of this Advisory Panel and is included as part of this report. 



This material is based upon work supported by the National Science Foundation under NSF contract 
number CSI-8850357. Any opinions, findings, and conclusions or recommendations expressed in this material are 
those of the authors and do not necessarily reflect the views of the National Science Foundation. 
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Interpretive Overview: 



A Statement From the Assessment's Advisory Panel 

Doris R. Helms (Principal Author), Associate Dean for Undergraduate Instruction, Clenison University 
Stuart B. Crampton, Department of Physics and Astronomy, Williams College 
Larry K. Monteith, Chancellor, North Carolina Stale University 
Suzanne W. Slaydcn, Department of Chemistry, George Mason University 
Thomas Tucker, Department of Mathematics. Colgate University 
David E. Wiley, Dean, School of Education and Social Policy, Northwestern University 



It would be difficult to imagine a project more 
successful than this one... All courses ir the program 
were reviewed and revised; new equipment has been 
introduced in general courses to sophomores, and then 
used in ten other courses in the department as well as 
senior seminar and master's projects. Formerly, they 
showed slides and used data from books; now, they 
use labs and field work... Tlxc program, according to 
the dean, is now a "flagship"... and the department has 
been given a new tenure stream faculty position... TJie 
dean cited the program as an important factor in 
receiving a recent $100,000 state grant to interest inner- 
city youth in science. 

Dr. Thomas Tucker, C ;ate University 

Site Visit Consultant ^ Advisory Panel Member 



Tfxis project is doing extremely well. Apart from its 
direct impact on several courses and student research 
participation, it has had a catalyst effect on several 
other activities in the department. ..biolog\> has created 
an impressive teaching/research environment for its 
majors. TJicre is a definite upbeat feeling among the 
faculty... Tliey are also generating an extraordinary 
number of students who continue on to do some 
graduate work in the field (25-30 per year) for a college 
with a total enrollment of 1,000. 

Dr. Norman Henderson, Oberlin College 
Site Visit Consultant 



These statements by CSIP site visit consultants convey 
the essence of what has made the College Science 
Instrumentation Program 1 a success. Based on our own 
experiences, and on a thorough review of information 
compiled by Westat, the CSIP Advisory Panel 
enthusiastically endorses this NSF effort, and predicts 
that the extended impact of the CSIP program will be to 



rejuvenate undergraduate science, mathematics and 
engineering education. 



Background 

That science, mathematics, and engineering education 
in this country face profound difficulties is no longer in 
doubt. The problems and consequences are numerous 
and well documented. Some examples from Science 
and Engineering Indicators: 

■ Since 1983, the number of students interested in 
pursuing degrees in science and engineering 
(S/E) has steadily declined. 

■ Of 5.7 million college students presently 
enrolled, only 780,000 are preparing for S/E 
careers; only half of these will graduate with S/E 
degrees; and oniy \% will continue to study for 
the Ph.D. 



CSIP is an attempt to counteract these disturbing trends 
by supporting updating and revitalizing of 
undergraduate laboratory curricula in science, 
mathematics, and engineering. As such, it is part of a 
larger NSF effort to support and advance science 
education, the overarching goal of which is to encourage 
young people to become the scientists and engineerings 
of tomorrow, and to ensure i' at our future leaders in 
business, education, and government are scientifically 
literate and informed tbout technological issues 
affecting our national and global welfare. 

Increased retention of undergraduates with S/E 
interests demands curriculum changes - new ways lo 
teach science that allow science to be experienced 
rather than learned from a book. According to recent 



The College Science Instrumentation Program (CSIP) was open 
only to non-doctoragc-granting institutions from 1985-87. In 1988 
the program was renamed the Instrumentation and Laboratory 
Improvement Program (ILI). and was opened also to two-year and 
doctorate-granting institutions. 
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findings from the National Assessment of Educational 
Progress (NAEP), creativity and curiosity must be 
emphasized through participatory classroom and 
laboratory activities if science proficiency and altitudes 
toward science are to improve.' 

CSIP Program Overview 

CSIP's fundamental premise is that laboratory work and 
field experience are at the heart of understanding how- 
science works and what scientists do. Because modern 
instrumentation is a vital component of "doing* science, 
it is key to the changes we must make in science 
instruction. 

For many colleges and universities, the last major 
expenditure of funds for undergraduate instructional 
science equipment occurred in the 1960's in response to 
the Sputnik challenge. Many college laboratories are 
now equipped with 30-year-old instruments that bear no 
operational resemblance to their modern counterparts, 
and which, in some cases, are unsafe. The problem of 
obsolescence has been exacerbated by an increased 
emphasis on research with less and less funding directed 
toward undergraduate instructional equipment. Small 
colleges have found it increasingly difficult to compete 
in the research market and, without available funds for 
instructional equipment, their greatest asset quality 
undergraduate educational experiences — is being 
compromised. 

Therefore, when NSF introduced the CSIP grant 
program in 1985, it focused on colleges with a primary 
mission of undergraduate instruction. CSIP offered 
these 2,706 non-doctorate-granting institutions an 
opportunity to acquire modern scientific 
instrumentation as part of the process of upgrading 
their science curricula, Proposals for CSIP grants were 
judged competitively. Awards were made for the 
purchase of scientific instrumentation, but the crucial 
factor in determining the merit of a proposal was the 
quality of the plan for improving instrument-based 
courses or curriculum. In all cases, investigators were 
required to demonstrate how requested instrumentation 
would be used to improve course work and student 
learning. 



Educational Testing Service. Ttw Science Report Card Elements of 
Risk and Recovery. Princeton, NJ: ETS. 1988. 
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CSIP Impacts 

Despite its modest scope during its first three sears 
(1985-87), CSIP has been a very successful program, 
and has had a considerable positive impact, as 
documented in the Weslal report. Although it is usually 
difficult to measure a program's effectiveness, especially 
when it purports to increase the quality of the learning 
experience or ol education in general, we believe lh;tt 
the wealth of statistical data and written comments that 
Weslal has analyzed with the help of the advisors panel 
provides clear evidence of wide ranging positive 
program impacts. Supporting the questionnaire data is 
evidence from the site visits, where a strong sense of 
excitement was fell by consultants who visited CSIP 
projects and witnessed the growth and improvement in 
programs and departments as a consequence of CSIP 
grants. 

Unlike more expensive national programs that lend to 
cater to those already effectively competing, CSIP 
provided opportunities for small colleges, minorits 
institutions, and college and universities that do not 
offer graduate degrees to develop new programs 
appropriate to local or regional settings, and to renew 
faculty interest in course development directly linked to 
student use of modern sophisticated equipment. As a 
result of CSIP funded programs: 

I. Students' laboratory skills and critical thinking 
skills improved and the use of computers lor 
interfacing, data collection and analysis 
increased. For example, awardees report: 

(Student) skills have improved with better 
tools. Tlie opportunities afforded by the 
new equipment automatically enhance the 
opportunities for intellecmal development... 
Motivation and attendance are both up. 

Since the level of the equipment used in 
training requires conceptual ability of the 
students, they are motivated to study and 
work harder, which in turn increases their 
skills, intellectual development and 
attendance. Student grades are climbing a 
bit more since their motivation and attitude 
toward the work load has improved. 

Impact on students appears substantial- 
excited, enthusiastic to work with research- 
level equipment, rather than just being told 
such equipment exists. 




Enrollments in science courses increased. 

Vie course is a product of innovative 
change within the department. The 
laboratory provides a hands on experience 
in molecular biolog\> that has resulted in 
improved skills, increased numbers of 
independent study projects, enrollment in 6. 
related courses is up, as are student 
presentations at national and regional 
meetings. 

We arc over-enrolled with a waiting list. 
Several using the advanced equipment 
have produced very good papers, and we 
have a strong flow of majors. 

More students participated in undergraduate 
research. Evidence from principal investigators: 

Almost all of our students are asking for 
opportunities to do undergraduate 
research. 

...we now have a higher level of expectation 
and accomplishment in undergraduate 
research. 

Students' familiarity with new laboratory 
technologies opened additional career 
opportunities in the sciences. 

7. 

Students tell us the course has been 
valuable in helpifig them find good 
positions upon graduation. One student 
was hired primarily because she was the 
only candidate for a laboratory position 
who had some computer interfacing 
experience. 

Anecdotal evidence suggests students now 
working in industry found experience on 
equipment helpful (especially to get job in 
first place). 

Faculty experienced increased opportunities for 
self development, recognition and reward. Both 
faculty and departmental morale increased in 
turn. It was reported: 

Access to this instrument has generated 
much pride and enthusiasm within the 
department as well as providing a state-of- 



the-art instrument for training 
undergraduates. 

Our enthusiasm about the project 
proposal, getting the award, and putting the 
project in place has not diminished. 

Equipment purchased for one purpose often 
became Ubed in a variety of ways not originally 
anticipated, fostering even greater cooperation 
among faculty, interaction between upper level 
and introductory instructors and integration of 
courses from ci variety of disciplines. From 
projects: 

As other faculty have obscn ed our integration of 
microcomputers into the classroom, they have 
become excited about possibilities for their 
courses. 

We are integrating the equipment 
throughout the curriculum from freshman 
to senior level courses. Originally, the 
project was proposed for upper level 
courses. 

Instruments purchased with these funds are 
being used in additional courses than 
originally proposed. Vicy have had a 
greater impact than originally proposed. 

It has revitalized undergraduate science 
instruction within the departments which have 
received grants, and the effects have often 
motivated changes in other science departments 
of the same colleges as well. Principal 
investigators report: 

We've had indirect impact on the rest of 
our facilities in that the equipment 
purchased for the project showed its 
capabilities and provided the model for the 
upgrade of the rest of the labs. 

Vie (project) is a model that has made a 
positive impact at all levels of the college. 
Vie curriculum interfaces with related 
disciplines and it has received full 
institutional support. 

Because I wanted a forum to share my 
thoughts on this course, problems 
encountered and solved, 1 organized a 
state-wide symposium on Biotechnology, 
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which attracted faculty from most of the 
(state) system (both Biolog\- and Chemistry 
departments represented). This has now 
evolved into a superb state-wide 
organization dealing with Biotechnology: 

8. The funds devoted lo CSIP have been well spent 
and in many cases, have leveraged several fold. 
Some evidence from projects: 

($10,300 grant) This project allowed us to 
initiate our computer science lab. The lab 
has doubled in size last year (with S1K000 
of additional equipment donation*/. 
Many, many new projects have started. M e 
also have a continuing lab budget. 

($y,000 grant) Our CSIP grant was 
evidence of program strength, and a 
contributor to space limitations involving 
laboratories. TJicse factors were important 
in obtaining a $3.5 million grant from the 
(private) Foundation for a new physical 
science building. 

In addition, there is evidence that the act of preparing a 
CSIP proposal, even one ultimately not funded, has 
beneficial effects, because the planning process clearly 
revealed the direction a department should take to 
upgrade its program; in many cases, support was found 
outside CSIP for these proposals. 

Future Steps 

CSIP is clearly a unique program that gives dedicated 
undergraduate teachers the tools they need and the 
recognition they deserve as they strive to implement the 
innovations they think appropriate to their 
environments. We recommend the following: 

1. Expand the CSIP program 

Undergraduate science, mathematics and engineering 
education would benefit enormously if the CSIP 
program were to become available on a wider basis. 
Extending eligibility to doctorate-granting institutions 
and two year colleges as part of the ILI program has 
already expanded its scope by two and one-half times, 
but CSIP/ILI should be funded and expanded further. 
Generalizing from the results found for CSIP, the 
doctorate-granting institutions should benefit as much 
as the type of schools supported by CSIP in 1985-1987. 
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2. Maintain interest in small undergraduate 
institutions 

Man\ of the impact on institutions reported in tin - 
CSIP evaluation uere related to the character of lhe.se 
schools. A large proportion of the CSIP granlee 
institutions were very small schools that never received 
grants for scientific purposes - particularly from the 
highly prestigious National Science Foundation. The 
impact in such small communities lends to reverberate 
at all levels, much more so than at larger and 
scientifically more sophisticated institutions. Since 
these small schools are a major source of students who 
pursue Ph.D.'s in the sciences, further expansion should 
not overlook the small undergraduate institutions wilh 
limited resources. 



3. Increase funding lor introductory courses 

In most institutions, for a variety of reasons, 
introductory laboratory classes are often the last lo be 
given resources and attention. Increased expansion and 
funding would allou for maximum impact at the 
introductory level. Large classes require many duplicate 
pieces of equipment, and because the equipment is used 
solely for leaching il is difficult to obtain through 
available funding sources. 

4. Maintain funding for upper-level courses and 
for undergraduate research 

Student participation in upper- level courses is also 
critical, as is active participation in research at this level. 
Student research is one of the best mechanisms for 
"doing" science, and students in these programs serve as 
models for others. It is also imperative that upper level 
or research faculty work with those teaching lower level 
courses. CSIP is a medium for this exchange as both 
groups become involved in curriculum change and 
renewed efforts to assess student learning. 

5. Increase funding for courses that serve general 
education students and promote science literacy 

If the program were to be expanded, large numbers of 
general education students in our colleges and 
universities would also be exposed to improved 
laboratory facilities where they could experiment and 
"do science" rather than just read about it. If we are to 
have a scientifically literate public, such opportunities 
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must be made available. CSIP/IL1 can do this if its 
scope and funding are increased. 

6. Increase networking and dissemination efforts 

Products of successful programs should be disseminated 
and networking of successful grantees with others 
should be undertaken immediately. The enthusiastic 
response to the CSIP program indicates that there is 
genuine interest among science faculty in the program 
goals. Now that many projects have been developed 
successfully, the value of the program could be greatly 
increased by disseminating descriptions of successful 
programs to others who seek to accomplish similar 
objectives in the most cost-effective manner. A modest 
investment in the necessary expansion of NSF staff for 
this purpose would be justified. 



Our future as a nation depends on turning the tides of 
science, mathematics and engineering education. 
CSIP/ILI is a successful model which has proven that 
this can be done by providing up-to-date equipment for 
stale of the art demonstrations and experiments that 
pique the students' curiositv and allow for hands-on 
problem solving in the true "spirit" of science. 
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executive Summary 



The College Science Instrumentation Program 
(CSIP) was developed by the National Science 
Foundation's Office of Undergraduate Science. 
Engineering and Mathematics Education (USEME) 
as a vehicle to provide seed money matching funds 
for the acquisition of laboratory instrumentation in 
projects to improve the quality of undergraduate 
science/engineering education. In its first three 
years of operation, 1985-87, the CSIP program 
awarded 780 competitive grants in the $5,000 to 
$50,000 range (totalling $19.7 million) for proposed 
curriculum improvement projects at non-doctorate- 
granting undergraduate institutions. Since then, the 
program has been expanded and renamed the 
Instrumentation and Laboratory Improvement (ILI) 
program. The expanded program is currently open 
to all two-year and four-year colleges and 
universities, both doctorate-granting and non- 
doctorate-granting, and it now awards grants of up 
to $100,000. 

This report is a wide-ranging, independent 
evaluation of the first three years of the CSIP/1LI 
program. The findings apply only to the kinds of 
institutions and grants that were involved at that 
time. 

The evaluation was conducted by Westat, Inc., under 
contract to NSF. In November 1988, evaluation 
study questionnaires were mailed to all 234 grantees 
in the 1985 CSIP and to samples of 100 grantees 
each in the 1986 and 1987 programs. In addition, 
samples of 125 unsuccessful CSIP applicants were 
selected in each of the first three program years, and 
they too were sent questionnaires asking about their 
experience in the program. Lastly, postsurvey site 
visits were made to a randomly selected subsample 
of 49 grantees to validate the questionnaire data and 
collect additional information about project 
development and impacts. 

All findings are in the form of program-wide 
estimates and are based on response rates of 90 
percent for grantees, 82 percent for unsuccessful 
applicants, and 100 percent for projects that were 
sampled for site visits. 



ERIC 



Findings 

Resource Development 

The CSIP program provided "seed money" intended 
to stimulate the upgrading and expansion of 
science /engineering instructional programs and 
resources at recipient institutions. One indirect 
indicator of program success is the amount of 
additional laboratory space that was created to 
house CSIP-gcncrated programs and equipment. 
About one-quarter of the grantees (26 percent) 
reported that they received additional laboratory 
space as a result of their CSIP projects. The amount 
of additional space averaged 640 square feet per 
affected project and was 130,000 square feet in total. 

Another quantitative indicator is the financial return 
on NSFs investment: the total dollar size of the 
projects that ultimately germinated from the CSIP 
seed money. If all grantee institutions honored their 
commitments to provide the equipment matching 
funds promised in their proposals, the total dollar 
amount of equipment in CSIP projects should be 
somewhat more than twice the amount of the CSIP 
grants. In fact, the projects spawned by CSIP were 
considerably larger than that in most cases (Figure 
1). 

Figure 1.- FinanciaJ composition of projects generated by CSIP 
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Overall. CSIP grants to ihc approximate!) 7(>(J 
grantees represented in the survey data totalled 
$17.9 million, an average of $23,500 per award. 
When institution matches, and overmatches, 
unanticipated vendor contributions and price 
increases, installation costs and estimated 
operating/maintenance costs over the first five years 
of the project arc added in, along with additional 
resources and equipment subsequent!) attracted b\ 
the project, the aggregate project size increases to 
$80 million, or an average of $105,000 in equipment 
and expenses per project. This represents a return 
of about 450 percent on NSFs investment. This is 
an extreme!) conservative estimate, howc\er. for 
four major reasons. 

First and most important, the return on investment 
calculation does not include any valuation of the 
lime spent by principal investigators (Pis) and other 
CSIP project staff in the development of the 
upgraded/expanded curricula in which the project- 
supported equipment was used. The Pis reported 
spending an average of 337 hours (42 S-hour days) 
apiece in CSIP project implementation activities, of 
which nearly half was spent specifically in curriculum 
development. 

Second, the reported 450 percent return on 
investment does not include any valuation of the 
130,000 square feel of additional undergraduate 
laboratory space created to house CSIP-generaled 
projects. 

Third, the report documents many instances where 
grantees vastly understated the extent of additional 
resources that wei e attracted to their departments 
and institutions as a result of their CSIP projects. 
For many of the small, teaching-oriented 
undergraduate institutions served by CSIP, the NSF 
grant was an important form of national recognition 
of their program, a certification of excellence that 
was valued in its own right and that also proved 
valuable in attracting additional funding support to 
the program from within the institution, from private 
foundations, from local industry, etc. Many 
examples of such resource leveraging were found, 
going far beyond the $15.8 million that was reported. 

Fourth, the return on investment calculation does 
not include the energizing, stimulating effect the 
CSIP program had upon those who applied for but 
did not receive CSIP grants. About three-fifths of 
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these unsuccessful applicants reported that the) 
were subsequently able to obtain funding support for 
at least some of the equipment the\ had hoped to 
get through CSIP. whether b) using the matching 
funds their institutions had alread) agreed to commit 
or by finding other sources of funding support. The 
total amount of this additional instructional 
equipment indirect!) generated b\ CSIP is estimated 
to be about S27 million. 

In vicv* of these considerations, it seems likel) that 
the real multiplier effect of the CSIP program in 
generating improved instructional equipment and 
resources in undergraduate science education, 
bcyor d what could be purchased with the modest 
amou.its actual!) invested b\ NSF. is much greater 
than 45U percent. Indeed, it is probably somewhere 
in the 700-°00 percent range overall. For many 
individual cases documented in this evaluation, the 
multiplier effect was even greater than that. 

Impacts on Students, Faculty, and 
Institutions 

Among projects that were mostly or fully operational 
with students at the lime of the evaluation, most 
were reported to be having significant educational 
impacts (Figure 2). Project-provided opportunities 
for students to have hands-on experience in 
understanding and working with advanced scientific 
equipment were frequent!) cited as contributing to 
observable improvements in students' skills (74 
percent of projects), motivation and attitude toward 
coursework (73 percent), and conceptual and 
intellectual development in the subjects affected (67 
percent). Nearly half of the projects had already led 
to the development and presentation of student 
papers using CSIP-funded equipment (43 percent), 
and almost a third claimed to have influenced 
students to select careers in science/engineering (30 
percent). 

The reported impacts on scientific and engineering 
pipeline factors — presentations, careers, 
enrollments was lower. However, given the brief 
duration of most projects at the time of the survey 
and the lime lag inherent in pipeline effects, this 
should not be surprising. 
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Figure 2.-- Percent of CSIP projects with positive Impacts on students 
in various areas 
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One important question addressed b\ the evaluation 
site visit teams in interviews with Pis. department 
heads, and others wa.s whether the grantee's project 
would have been able to go forward in the absense 
of CSIP. The site visitors' answer, in most cases, was 
"no," at least not with the size and impact it achieved 
through CSIP (Figure 3). Only an estimated 15 
percent of CSIP-funded projects would have been 
mostly or fully implemented with other funds, had 
the CSIP gram not been received. This suggests thai 
it was not just a matter of CSIP aligning itself with 
meritorious projects that were destined for success, 
with or without the program. In most cases, it was 
the judgment of the PI and the independent 
evalualors that the CSIP program itself was 
instrumental in the creation and actualization of the 
project. 



Figure 3.-- Site visit assessment of whether projects would 
hsve occurred without CSIP 



The findings from an extensive series of post -survey 
site visits generally confirmed these Pi-based claims 
of significant educational impacts. Insofar as the site 
visit teams disputed the Pi-provided assessments, the 
complaint was usually that the Pis' questionnaire 
responses had understated the educational impacts 
of the projects. 

Two other areas where both the Pis and the site visit 
teams agreed that CSIP projects were consistently 
having strong positive impacts were: 

■ Improved morale of department faculty, 
including the PI (71 percent of operational 
projects, according to the site visitors); and 

■ Increased prestige of the recipient 
department within the institution (88 percent 
of projects). 

In the areas of faculty morale and department 
prestige, the value of the CSIP grant as an all-too- 
rare expression of national recognition and 
encouragement of excellence in undergraduate 
education may often have been as important as the 
intrinsic educational value of the equipment and 
curriculum changes the grant funded. 
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Program Reach 

In its first three years of operation, the CSIP 

program attracted a total of 3,226 proposals from 

2,449 different principal investigators (Pis) 

representing 811 undergraduate colleges and 
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universities. The program awarded a lolul ol 7su 
granls 10 7o2 different Pis from 4UJ diflerenl 
institutions during thi.s period. The award?* totalled 
$1 C J.7 million and a\eraged S25.224 per project, 
which is aboul ihe middle of the $5,001) lo S.MJ.UOU 
applicable range during his period 4 

Awards were distributed across fields in proportion 
lo the number of proposals received. Disciplines 
with the heaviest representation were chcmislry (27 
percent of the awards), biology (21 percent), 
engineering (12 percent), and physics/astronomy ( 12 
percent). The six remaining discipline categories 
(computer science, earth science, mathematics, 
psychology, social sciences, and interdisciplinary ) 
each received under 10 percent of the total awards. 

Half of all awards were lor projects that would affect 
both upper division and lower division 
undergraduates. Most of the rest (45 percent of the 
total) were targeted only for upper division students. 

Most grantees proposed lo address more than one of 
CSIP's four program objectives: 

■ 95 percent sought "introduction of modern 
instruments lo improve the experience ol 
undergraduate students in S/E courses, 
laboratories, and field work"; 

■ 58 percent involved "interfacing of computers 
with scientific instrumentation and other 
appropriate uses of current technology in 
S/E instruction"; 

■ 59 percent entailed "development of new 
ways of using instrumentation to extend 
instructional capabilities"; and 

■ 0 percent proposed the "establishment of 
equipment sharing capability via consortia or 
centers." 



the e\alualion began. C)\crall. So percent ol the 
NS5-S7 grantees reported that they were mostly or 
fully operational: this included 9S percent ol the 
I Wo giuniccs. SN percent of the grantees, and 
7h percent ol the grantees. In the same \ein. 

over three-fourths of all grantees reported that their 
piojects were on schedule or ahead of schedule. 
Almost all grantees ( ( >S percent) reported lhat the 
equipment ihey purchased was the same as. oi 
functionally equivalent lo. the equipment they 
originally proposed lo acquire. Validation data from 
the site visits support these claims. 

Thi> i> not lo say lhat grantees have not had any 
problems implementing their projects. On the 
contrary, almost hall (44 percent) of all grantees 
encountered significant problems oi one kind or 
another as they sought lo develop their CSIP 
projects. As reported by 20 percent of the grantees, 
the most common problem was thai I heir projects 
were delayed by unexpected difficulties in acquiring, 
installing or learning to use their equipment. 
Problems in financing or in arranging adequate 
maintenance lor the equipment were reported by 12 
percent of all grantees. 

Additionally, many grantees (40 percent ) reported 
thai project setup activities look significantly more 
lime than they had expecled. These activities 
required an average (mean) of 336 hours per 
grantees or over 40 person-days of lime-on-lask 
work. Few grantees (19 percent) received any 
release time for this work. 

in view of ihe magnitude of the problems and 
challenges encountered by grantees, ihe large 
amounts of lime typically required for project seiup. 
and the fact that NSF has nol closely monitored 
grant progress. CSIP grantees as a group have 
demonstrated remarkable resolve, resourcefulness, 
and conscientiousness in doing what needed lo be 
done lo overcome obstacles and adhere lo their 
proposed project schedules and specifications. 



Project Development and Implementation 

Grantee projects in all three program years were 
generally further along than had been expecled when 



4 In the case of Pis who received multiple CSIP awards in the 
1985-87 period, the evaluation study covered only ihe first 
award. 



Program Administration 

From all indications, the 1^85-87 CSIP program was 
administered in an extraordinarily scrupulous and 
fair-minded manner. Granls were distributed widely 
among institutions; they were distributed among 
disciplines in proportion to the number of proposal 
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received; and ihe only individual or institution 
characteristic that was predictive of success in 
winning CSIP awards was persistence in applying 
more than once. 

Grantees and unsuccessful applicants alike had few 
criticisms of the way the program has been 
administered by NSF. Some complaints were made 
about the liming and distribution ol the program 
announcements, about the (loo short) deadline for 
proposal submission, and about the clarity c.f the 
program guidelines. Even among unsiirtv-^ful 
applicants, however, such complaints were 
infrequent (all involved fewer than 3D percent of 
unsuccessful applicants and fewer than 20 percent of 
grantees). 

The one area where complaints were common was 
"clarity of feedback from proposal review." which 
was identified by half of all unsuccessful applicants 
(51 percent) as an aspect of the program needing 
improvement. Judging from the comments 
accompanying this response, clarity per se was not 
really the issue in many cases. Many of the 
unsuccessful applicants had invested considerable 
effort and hope (and. sometimes, personal prestige 
within their departments and institutions) in the 
preparation of their proposals, and some of them 
took the proposal rejection letter and accompanying 
critical feedback from NSF as a more general 
rejection of themselves personally, of the quality ol 
their department's leaching program, or even of the 
quality of their institution. This propensity is 
something NSF should be aware of and extremely 
sensitive about in the future. 



Conclusions 

In its first three years, the CSIP program was 
remarkably successful, demonstrably revitalizing and 
enriching the instructional programs at many 
recipient institutions and producing a high return on 
NSFs financial investmeni. The very modest 
investments represented by most CSIP grants during 
this period often had remarkable impacts on 
recipient departments and institutions, impacts that 
sometimes seemed far out of proportion to ihe size 
of the grant. It was not uncommon for new courses, 
new laboratories, even new degree programs to be 
created as a result of $10,000 or $20,000 CSIP 
grants. The site visit teams often saw evidence of 



genuine revitalizalion of leaching programs and 
renewed enthusiasm of faculty, as well of increased 
student interest and involvmcnl in the curriculum. 
In addition in lhe.se educational impacts, the 
existence ol the NSF grant often attracted additional 
interest to recipient programs - and additional 
funding support from within the institution and 
also from outside sources. 

In trying to account for the seemingly out sized, 
disproportionate impacts documented in this 
assessment, several factors appear to have been at 
work: 

■ Most CSIP- funded projects were highly 
meritorious (needed, worthwhile) and were 
conscientiously, competently administered by 
Pis eager at the chance to put their ideas for 
curriculum improvement into effect; 

■ CSIP was one of very few Federal programs 
that recognize and reward excellence in 
undergraduate science leaching in a 
meaningful way, which made ihe recognition 
especially salient and significant at many 
institutions; 

■ Faculty and departments at the 
i ndcrgraduale (non-doctoraie-granting) 
institutions thai were the focus of CSIP 
during the 1985-87 period have comparatively 
few opportunities for external recognition or 
support of any kind, not just in the area of 
leaching; most are not actively* involved in 
grant-supported research, and some have not 
had any Federal research grants for years; 
and 

■ Partly because ihe faculty and departments al 
ihese institutions do not have access lo grant- 
funded research equipment (which could be 
used partly for teaching or could be 
convened to instructional use when no longer 
needed for research), it is especially difficult 
for them to keep their instructional 
equipment reasonably up to date. 

The combination of these four factors may account, 
in part, for the impressive impacts of the CSIP grant 
program at non-doctorate-granting undergraduate 
institutions. Now that the CS1P/IL1 program has 
been expanded to include doctorate-granting and 
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two-year institutions, il will be interesting to sec 
whether the kinds of program impacts that occurred 
during the first three years of the program will also 
be manifest in these other types of sellings. 
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INTRODUCTION 



Background 

The College Science Instrumentation Program 
(CSIP) was developed by the National Science 
Foundation (NSF) in response to problems 
predominantly undergraduate schools were having 
keeping pace with current developments in the 
sciences. Specifically, the lack of adequate funds to 
purchase up-to-date laboratory equipment kept 
many from initialing significant curriculum 
improvements. In a competitive grant program 
begun in 1985, CSIP awarded matching equipment 
grants to school:; with plans to improve 
undergraduate science instruction. Requests for 
CSIP equipment funds were evaluated by NSF on 
the basis of the curriculum development plans 
proposed. 

The CSIP program promoted the long-term goals of 
NSF and the scientific community of increasing the 
number of undergraduates who choose science as a 
career, and enhancing the preparation of these 
students. Primary objectives of the program were to 
support: 

■ The introduction of modern instruments to 
improve the experiences of undergraduate 
students in science and engineering courses, 
laboratories, and field work; 

■ The interfacing of computers with scientific 
instrumentation and other appropriate uses of 
current technology in science and engineering 
instruction; 

■ The development of new ways of using 
instrumentation to extend instructional 
capabilities; and 

■ The establishment of equipment sharing 
capability via consortia or centers. 

Additional objectives of the program were to 
support: 

■ Projects that set standards for instrumentation 
and its use against which other institutions 
measure themselves and which they strive to 
achieve; and 

■ Products such as laboratory manuals and other 
scholarly publications serving a common goal as 
well as local improvement. 



In addition to its focus on enhancing the experiences 
of science majors in inst. uctional sellings, the CSIP 
program was also intended to impact other student 
populations. Target groups include sludenls 
interested in undergraduate student research, non- 
science majors, students training to become 
precollcgc science and mathematics teachers, and 
women, minorities, the physically disabled, and other 
groups traditionally underreprcsenlcd in science and 
engineering careers. 



The Dimensions of the CSIP Program 

In its first three years of operation, the CSIP 
program awarded 780 grants to 410 different 
institutions (out of a total of 2,706 non-docloratc- 
granting colleges and universities eligible to 
participate). A total of $19.7 million was awarded in 
those three years for the purchase of laboratory 
equipment, with individual grants ranging from 
$5,000 to $50,000. 5 Table M shows the number of 
proposals received in each year, the number granted, 
and the ratio of awards to proposals received. 

Table 1-1. CSIP proposals by grant year: 1985-87 



Grant year 



Proposals 


Total 


1985 


1986 


1987 


Proposals received 


3.226 


1,335 


922 


969 


Grants awarded 


780 


234 


211 


335 


Award rate 


24% 


18% 


23% 


35% 



Source: Assessment ol the 1985-87 College Science Instrumentation 
Program, Office of Studies and Program Assessment, NSF, 
1990 



The number of proposals submitted to NSF in 1986 
and 1987 leveled off in the 900s after the initial wave 
of 1,335'received in 1985. Increased funding for the 
1987 grant year permitted a greater number of 
awards in that year than in the two previous years. 
Together, these factors resulted in increasing success 
rates for applicants over time. 

As shown in Table 1-2, 30 percent of the 2,706 
eligible schools submitted a CSIP proposal in at least 
one of the program's first three years. Over time, 
from a first-year high of almost 25 percent, the 



The CSIP program ha? since been expanded and renamed the 
Instrumentation and Laboratory Improvement (ILI) program. 
The expanded program i: open to a wider range or 
undergraduate institutions, and the grant ceiling is now 
$100,000. 




proportion submitting dropped to about IS percent 
in both and 1987. The overall success rale for 
schools submitting at least one proposal during thai 
lime was over 50 percent. 



Table 1-2 Rates of CSIP participation and success for 
eligible institutions by grant year. 1985-87 
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CSIP proposals .. 




30 0 


24 8 


18.8 


18.3 


Of schools submitting 












proposals, percent with 










one or more funded. 




506 


28 4 


34.8 


OA 



Source. Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990 



Assessment of the Effectiveness of the CSIP 
Program 

In March 19SS, NSF awarded a contract to Wcslat, 
Inc., of Rockvillc, Maryland, to assess the 
effectiveness of the CSIP program during its first 
three years of operation. This report presents the 
principal findings based on data collected through 
mail questionnaires completed by CSIP grantees and 
unsuccessful applicants, and site visits to CSIP 
projects. All reported data are weighted to 
represent the CSIP universe. This report is 
organized around two broad topic areas: 

■ CSIP program impacts (Chapters 1-4); and 

■ CSIP program characteristics (Chapters 5-7). 

For those interested in further breakouts of the 
questionnaire data, a report entitled "Detailed 
Statistical Tables" is available from NSF. For more 
information on sampling and data collection 
procedures, a third volume, entitled "Technical 
Report," is available. 



questionnaires and in the interpretation and 
presentation of the findings. The Panel was also 
asked to review the assessment findings and to write 
the interpretive overview of the CSIP program that 
appears at the front of this report. Members of the 
Panel are identified in the Acknowledgments section 
of this report. 

Data Sources 

Data for this assessment of the CSIP program were 
collected through mail questionnaires addressed to 
principal investigators (Pis) of funded CSIP projects, 
and to proposed Pis of unfunded projects. 
Additional data were obtained through site visits to a 
subsamplc of operating projects. Copies of the 
grantee and unsuccessful applicant questionnaires 
appear in Appendix B, as does the site visit summary 
form. 

Sampling for the mail questionnaires was designed 
to achieve adequate representation by year, field of 
science, the success and participation rate of the 
submitting institution, and whether the proposed PI 
is a new or a previous applicant. 

Among grantees, 434 projects were sampled, and Pis 
from 391 responded, resulting in a response rate of 
over 90 percent. Among unsuccessful applicants, 
375 were sampled and 311 responded, for a response 
rate of 83 percent. Site visits were made by two- 
person teams to a subsample of 49 CSIP projects. 
The current PI, other faculty, and administrative 
figures were interviewed information obtained was 
used to complete site visit summary forms, which 
distilled the information into a quantitatively 
manageable form. 
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Assessment Advisory Panel 

An Assessment Advisory Panel, composed of six 
academic scientists and engineers familiar with the 
CSIP program, was appointed to provide technical 
oversight and assistance in the design of the 
assessment and the development of the 
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PART 1 
PROGRAM IMPACTS 
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Chapter l 
CSIP Project Resource Development 



Highlights 

This chapter describes the extent to which projects 
have leveraged resources beyond the original grant 
award. Data presented are from the grantee 
questionnaire and represent Pi-provided 
information about their projects. 

In the 1985-87 period, CSIP grants averaged $23,500. 
The program required a one-to-one match from 
institutions, but the assessment found that most 
projects leveraged substantially more resources than 
this. In fact, when institution matches, overmatches, 
and implementation costs were added to vendor 
donations, and additional resources and equipment 
subsequently obtained, the average project value 
rose to $105,000--a return-on-investment of about 
450 percent for the CSIP program. This measured 
return, impressive by any standard, is a minimum 
estimate. It does not include any valuation for the 
130,000 square feet of additional space generated by 
CSIP projects, the 337 hours of labor required, on 
average, to get projects off the ground, or any 
correction for what appears to have been significant 
underreporting by grantees of the additional 
resources their grants have helped leverage. 

Above-Match Resources Obtained 

CSIP grantees were required to match the amount 
of their NSF grant with instrumentation funding 
from other sources. Most projects obtained 
financial support well beyond this minimum level, 
leveraging additional resources by way of substantial 
overmatches, discounts, upgrades, and equipment 
donations (Table 1-1). NSF contributed an 
estimated $18 million over the 3 years covered in this 
assessment, 1 for an average of about $23,500 per 
project. Matching funds for these projects total over 
$22 million, or $28,800 per project; an average 
overmatch of $3,300. Furthermore, over a third of 
the projects (35%) received more support from their 



This and other totals reported in this chapter slightly 
underestimate true program amounts as a result of the 
urduplication procedure used in this evaluation. There were 
19 grantees during the 1985-87 period who received more than 
one CSIP award. They were surveyed only once, with reference 
to their first award, and consequently the second award is not 
represented in the survey findings. Additionally, 2 of the 762 
grantees who are represented submitted questionnaires thai 
did not disclose the requested financial information. They, too, 
arc not included in the Table 1-1 estimates. 



equipment vendors than had been planned in the 
CSIP proposal. These vendor contributions totalled 
an estimated $5 million over and above whatever 
discounts had been negotiated when the proposal 
was submitted. Often, especially when changes in 
equipment prices or offerings occurred between the 
proposal and the award, vendors helped by providing 
deep discounts so the PI would be able to obtain the 
originally planned equipment at the earlier (now 
obsolete) price or by providing an upgraded model 
at no extra charge. Also, it was not uncommon for 
Pis to persuade vendors to donate additional 
equipment, ove^ and above what had been agreed, 
once NSF had certified (by virtue of its grant) that 
the Pi's project was indeed meritorious. 

Such vendor contributions did not always occur, 
however. For 21 percent of the projects, it was 
necessary for the institution to spend more money 
than had been planned to acquire the basic project 
equipment. These unanticipated additional costs, 
which usually reflected vendor price increases since 
the proposal, totalled over $1.3 million, for an 
average of $8,300 per affected project. Because they 
were unanticipated, and unbudgeted, these 
additional costs often represented a significant 
hardship for the institution. 

Cumulatively, these sources contributed a total of 
over $46 million of what could be considered base 
equipment for CSIP projects, amounting to more 
than 2.5 times the NSF investment. 



Costs of Project Implementation 

For most CSIP grantees, project implementation 
required expenditures that could not be covered by 
the NSF contribution or matching funds. 
Ninety percent of projects required such additional 
funds for necessities such as facility renovation and 
equipment installation, supplies, utilities, equipment 
maintenance, and salaries for work related to 
installation and maintenance of the equipment. The 
projects that reported expenses in these areas are 
estimated to spend $18 million, or an average of 
$26,200 per project, for project implementation over 
the first five years of operation. Like the 
unanticipated price increases discussed above, some 
institutions did not fully anticipate project operating 
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Table 1-1. Principal investigator's report of cost components for CS!P projects: 1985-87 



Project cost componeni 


Projects affected 


Dollar amount 
(in thousands) 


Number 


Percent 


Total 


Mean 


Equipment for base project 










Toial 


760 * 




46 1 1 4 


60.7 


NSF contribution (CSlP grant) 


760 


1 on 


1 7 874 


23.5 


Required match and planned overmatch 


760 


100 


21 .91 7 


28.8 


From institution 


714 


94 


17.820 


24.9 


From other source 


158 


21 


A.098 


25.9 


Other overmatch 










Total 


372 


49 


6.323 


17.0 


Additional cost for base equipment 










(e.g., due to price increase since proposal) 


1 57 


c. 1 


1 307 


ft 


Vendor contribution 










Totat 


268 




5.017 


18.7 


Deep discounts on price 


185 


24 


3.282 


17.8 


Upgrades 


76 


10 


345 


4.6 


Other (e.g.. additional equipment) 


50 


7 


1.389 


27.5 


Installation and operating costs, first 5 years 










Total 


688 


91 


18.019 


26.2 


Renovation and installation 


361 


48 


2,630 


7.3 


Annua! maintenance (x 5) 


458 


60 


5,751 


126 




586 


77 


2,892 


4.9 


Pro-rated salaries (technicians, students, faculty, release time) 












210 


28 


6,746 


32.1 


Other resources attracted as result of qrant 










Total 


292 


38 


15,645 


54.3 




242 


32 


9.830 


40.6 




100 


13 


6,015 


60.2 


AJI reported components 












760 


100 


79,978 


105.2 


Return on NSF Investment (total divided by NSF contnbution) 






447% 





•Estimates are based on 760 of the 781 projects funded tn 1985-87. which account for $17.9 million of the $19.7 million awarded dunng this period. 
For individuals with multiple awards, only the first one is counted. 

Source: Assessment of the 1 985-87 College Science Instrumentation Program, Office of Studies and Program Assessment. NSF, 1 990. 
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costs when they submitted their CSIP proposals, and 
that oversight sometimes presented real hardships 
for them. 



but is progressing and will result in 1 classroom* 1 
animal housing room, 1 animal lab, 3 human 
research rooms, and 3 faculty- offices. 



Additional Resources Attracted by the Grant 

While unanticipated price increases and operating 
expenses sometimes presented unexpected hardships 
for institutions, many projects also produced 
unexpected benefits of substantial magnitude, The 
seed money provided by CSIP often attracted 
additional resources that expanded projects well 
beyond their originally conceived scope and created 
additional capabilities for departments and 
institutions. Such expansions, totalling $15.8 million 
in additional resources, were reported by 38 percent 
of the 1985-87 projects. 

The following are examples of the kinds of snowball- 
like "multiplier effects" reported by grantees as 
having occurred once they . received their CSIP 
grants. The statements, provided bv the Pis, are 
preceded by parenthetical notations of the size of the 
NSF grant. 

■ ($9,000 grant) TJic institution, since the CSIP 
award, has upgraded all microscopy, including 
electron microscopy. Over the past 3 years, that 
amount is about $200,000. 

■ ($6,700 grant) {corporation} gave $20,000, 
{local} Foundation gave $30,000, {local} 
Consortium of Higher Education gave $14,000, for 
computers, video development, and workshop 
funds. 

■ ($44,500 grant) We received {$160,000} funding 
from the {private} Foundation for FT-NMR and 
GC-MS subsequent to our CSIP grant, in part, we 
believe, because of NSF's support of our 
laboratory 1 program. 

m ($11,500 grant) $50,000 challenge grant from 
private industry {was used} to improve lab 
facilities housing the instrumentation, plus $5,000 
in {corporation} and NSF grants for 
undergraduate summer research. 

■ ($15,000 grant) Institution began {$24,000} 
development of unused space into labs and 
classroom. Hxis development is not yet complete 



($7,100 grant) Partly because of the CSIP gram 
the geography/geology department is viewed by the 
university administration as a dynamic entity 
worthy of additional support. Tliereforc, they 
provided special funds {$3,100} for us to buy a 
personal computer, a 3-pcn plotter, and a digitizer. 

($41,000 grant) Just received a foundation grant 
of $50,000 (in addition to the first $50,000 from 
{corporation matching funds}) to add an 
additional lab (computer equipment, etc.). 

($10,300 grant) TJiis project allowed us to initiate 
our computer science lab. Hie lab has doubled in 
size last year {with $18,000 of additional 
equipment donations}. Many, many new projects 
have staned. We also have a continuing lab 
budget. 

($23,000 grant) {Corporation} funded a computer 
equipment grant partially because we already had 
a working microcomputer lab. We were given 8 
more micro's, a mini and all the networking 
equipment, and 2 laser printers {worth $100,000}. 

($27,400 grant) {We received} two state 
Tech/Engineering grants {which totaled} over 
$60,000, and a donated FT/NMR (worth over 
$80,000), largely as a result of the quality of our 
instrumentation and facilities begun with NSF 
grant. 

($20,000 grant) After some smaller donations, 
$14,000 in upgrades, and deep discounts, 
{corporation} supported our project with a 
massive equipment donation {$250,000}, enabling 
us to finish the basic lab and expand to a second 
'computing classroom. 1 



These examples indicate that there were many 
situations where Pis gave CSIP credit for attracting 
significant additional resources beyond the initial 
grant. However, the data contain many other 
instances where it appeared the PI could have 
claimed even more in the way of CSIP-leveraged 
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resources. Some examples of apparent under- 
counting follow. 

■ ($9,000 grant) Our CSIP grant was evidence of 
program strength, and a contributor to space 
limitations involving laboratories. Tlicsc factors 
were important in obtaining a $3.5 million grant 
from the {private} Foundation for a new physical 
science building, (no additional resources were 
claimed) 

■ ($27,000 grant) Definitely a contributing factor in 
our getting $494,000 ($200,000 in equipment) from 
a corporation for initiating a center for applied 
optics studies, (only the $200,000 in equipment 
was claimed) 

■ ($24,000 grant) Provided incentive to form a 
School of Science Biotechnology' Facility 
(interdepartmental between Chemistry and 
Biology) and money to initiate equipment 
allocations, (only $15,000 of initial allocations 
were claimed) 

■ ($35,000 grant) We were able to raise funds to 
allow us to add chemical ionization mass 
spectroscopy and low temperature GC capability 
to our original package. We are now the lead 
institution in a project sponsored by (he {private} 
Foundation to bring modern experiments to 
physical chemistry laboratories. We are also 
participants in a grant from the {second} 
Foundation to the college, (no dollar amount was 
claimed for these additional resources) 

■ ($50,000 grant) A {supplemental $83,000 
corporation} grant was followed by additional 
requests resulting in $190,962 from the {same 
corporation} University 1 Relations Board. 
Additionally, {a second} Foundation was 
convinced to contribute $500,000, with more than 
$250,000 being earmarked for electrical 
engineering labs, (only $450,000 was claimed, 
$324,000 less than the sum of the amounts 
mentioned) 

■ ($20,200 grant) Vie college received a grant from 
the {private} Charitable Trust of $994,000, and 
approximately $100,000 went to the Chemistry 
Department for equipment, and student and 
faculty research stipends, (only the $100,000 was 
claimed, not the full $994,000) 
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■ ($38,000 grant) University has provided additional 
equipment to the cell biology program (centrifuges, 
analytical balance, etc.) which has greatly 
expanded the utilization of moicriah provided by 
the CSIP grant; university t loney also provided 
supplies and lab tech salaries through a research 
grant, (only $10,000 in equipment was claimed: 
the grant was not) 

■ ($5,900 grant) Tixe college donated an additional 
4 Macintosh plus computers and a printer, and 
upgraded 3 of our original Mac 512\s to 
Macpluscs, It also responded favorably to an 
internal funding request for 8 computers for upper 
level labs, (the claimed amount for all of this, 
$6,000, seems too low) 

■ ($49,300 grant) Vic presence of the equipment 
has facilitated several faculty research grants, and 
the college has purchased additional equipment to 
supplement what we bought on the grant: 2 
centrifuges, HPLC, Elisa, spectrofluromctcr. 
($65,000 in equipment was claimed, but the 
research grants were not) 

■ ($8,700 grant) {We have received} a new science 
building addition (state funding) and $20,000 in 
equipment (we planned for a computer lab in our 
new facility in conjunction with our grant), (only 
the $20,000 equipment was claimed, not the 
building add-on) 

■ ($28,800 grant) Hie PI negotiated a long-tcnn 
loan of an electron microprobe from 
{corporation} ($150,000). Vic Co-investigator 
received a separate NSF-ILI grant for purchase of 
a new SEM (dollar amount unknown), (the 
microprobe was claimed, but not the SEM) 

These and many other examples indicate that some 
CSIP grantees may have been too modest wh^n 
assessing the effective use of their grants in 
attracting additional resources for their institutions. 
We can appreciate that a PI might think it immodest 
to claim that the recognition and credibility 
bestowed by a $9,000 CSIP grant was what made the 
difference in persuading a large foundation to fund a 
$3.5 million building at the college. But such an 
explanation is not implausible in the case of a small 
non-doctorate-granting institution that had not had a 
Federal grant of any kind for the 20 years preceding 
the CSIP award. 
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The question of whether grantee questionnaires 
adequately account for leveraged resources beyond 
the initial grant is explored further in Chapter 3. 
which presents the results of Mic visits at selected 
institutions. At this point, however, suffice to say 
that the questionnaire-based estimate of $15.8 
million in additional resources leveraged from CSIP 
projects appears too low. 



Leveraging of Additional Space 

In addition to further financial support, 200 grantees 
(one-quarter of the 1985-87 total) reported receiving 
additional space as a direct result of their CSIP 
project. About 130,000 total square feet was 
obtained, with an average of 640 square feet per 
affected project. It is difficult to place a dollar value 
on this space, and we have not attempted to do so, 
although it is apparent that lab space docs have 



value and that CSIP-generated space is part of the 
return-on-investmcnt for which NSF and its grantees 
deserve credit. 



Curriculum Development 

Another resource of substantial value leveraged by 
the CSIP program but not reported here in dollar 
amounts is the labor required integrate project 
equipment into the curriculum. Activities such as 
ordering, installing, and learning to use the 
equipment, developing new course materials, 
training staff, and integrating the equipment into the 
curriculum arc essential to the curriculum 
improvements expected in the wake of CSIP grants. 
Principal investigators are estimated to have spent 
an average of 337 hours on project setup, none of 
which was paid for by CSIP grants. This subject is 
discussed in more detail in Chapter 6 of this report. 
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Chapter 2 

The Impact of CSIP Projects on Students, Faculty, and Institutions 



Highlights 

This chapter focuses on the impacts operational 
CSIP projects have had on students, faculty, 
departments, and institutions. Because these 
impacts tend to be greater the longer projects have 
been in operation, most findings are broken out by 
CSIP program year. Data presented are from the 
grantee questionnaire and represent Pi-provided 
judgments. 



Table 2-1. Principal investigator's rating of the success of 
CSIP project by grant year: 1985-87 







Gran 


year 




Rating 












Total 


1985 


1986 


1987 


Total 


100.0% 


1 00.0% 


100.0% 


1 00.0% 


High success 


67.5 


77.3 


69.2 


59.3 


Moderate success 


27.6 


20.3 


27.5 


33.0 


Other 


4.9 


2.4 


3.3 


7.7 



Especially strong positive student impacts were 
found in areas related to training of future scientists 
(including motivation and attitudes, career choices, 
and the presentation of papers), and these affected a 
broad range of the student population. Positive, 
energizing impacts on faculty were also found. 
However, only limited evidence was found of project 
impacts beyond the institutions, either through 
equipment sharing or dissemination of results. 
Additional encouragement of such activities would 
be a worthwhile future focus for the CSIP program. 

Overall Project Success 

According to Pis, most projects have accomplished 
what they set out to do in their proposals: when 
asked to give a success rating, almost all said their 
projects were either highly or moderately successful 
in this regard (Table 2-1). Only 2 percent indicated 
major problems, and about 3 percent (mostly 1987 
grantees) indicated that it was "too soon to tell." The 
data suggest that success in achieving project 
objectives takes time: those that have been in 
operation longer are more likely to be rated as 
highly successful by Pis. More than three-quarters 
of 1985 grantees have projects they describe as 
highly successful, as do 69 percent of 1986 grantees, 
and 59 percent of 1987 grantees. 



Source: Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990. 



A related measure of the success of the CSIP 
program is whether the projects it has funded 
actively promote the objectives of the program. 
Respondents were presented with these CSIP 
objectives and asked to report which their projects 
exemplify: 

■ Introduction of modern instruments to improve 
the experiences of undergraduate students in 
science and engineering courses; 

■ Interfacing of computers with scientific 
instrumentation and other appropriate uses of 
current technology in science and engineering 
instruction; 

■ Development of new ways of using 
instrumentation to extend instructional 
capabilities; and 

■ Establishment of equipment sharing capability 
via consortia or centers. 

Almost all projects report that they involve the 
introduction of modern instruments, the overarching 
objective of the program. Fewer, but still a majority, 
entail interfacing computers with scientific 
instrumentation, and new ways of using equipment. 
And, only about 6 percent of projects have 
established or intend to establish equipment sharing 
consortia or centers (Table 2-2). 
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Table 2-2. Principal investigator's report of the CSIP program 
objectives project exemplifies by grant year: 
1985-87 



Objective 


Grant year 


1985 


1986 


1987 


Total 


100 O^c 


ioon<y A 1000% 


Introduce modern instruments 


96.6 


98 9 


91.3 


Interlace computers/instruments 


55.8 


604 


57.6 


New ways of using equipment 


59 1 


53.8 


63.0 


Share equipment/consortia 


5.3 


5.5 


6.5 



Note. Percents add to more than 100 because respondents could 
indicate more than one objective. 



Source: Assessment of tne 1985-87 College Science Instrumentation 
Program, Office of Studies and Program Assessment. NSF, 
1990. 



The figures for interfacing computers with scientific 
instrumentation, though respectable, are probabl} as 
low as they arc because relatively few computer 
science or mathematics projects promote this goal. 
This activity is not often relevant to these fields, 
since it would mainly be used to gather data from 
experiments. The surprisingly high percent of 
projects that promote new ways of using equipment 
are projects that involve innovation, the development 
of new methods and techniques for instruction, 
which, it is hoped, will result in improvements 
beyond the isolated department. 

Equipment sharing consortia or centers is the least 
often mentioned of the goals, and the lack of trend 
by year suggests that this goal is not one that is 
adopted late in the project. It seems that more 
could be done to encourage efforts toward this goal. 



Figure 2-1: Percent of CSIP projects with positive impacts on students in various areas 


Skills 
















73.9% 


















Motivation and attitude 




72.5% 


Conceptual or intellectual 
development 




67.0% 


Presentation of papers 












42.6% 






















Career choices 










30.5% 






















Increased enrollments 




22.4% 




















Attendance 




13.3% 










Grades 




7.8% 












1 
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50 
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Impacts on Students 

Ultimately, of course, the success of projects, and of 
the CS1P program itself, should be measured by the 
impacts on students, CSIP projects deserve high 
praise here. As depicted in Figure 2-1, in several 
specific areas the Pis of operational projects 
reported substantial positive impacts on students 
(see also Table 2-3): 

■ A clear majority in each year reported 
improvements in skills, motivation and attitudes, 
and conceptual or intellectual development of 
students; 

■ Many 19S6 and 1987 projects and 57 percent of 
1985 projects reported improvement in 
presentation of papers; 

■ About a quarter of 1986 and 1987 projects and 
42 percent of 1985 projects reported an increase 
in the choice of science careers; and 

■ About one in five projects reported an increase 
in science enrollments. 



Impacts on these six areas are essential for 
improvement in scientific understanding and for the 
promotion of the professional development of 
fledgling scientists. It is encouraging for the future 
that in each of these areas the figures are higher for 
projects that have been in existence longer. The 
areas of grades and attendance, which arc less 
crucial areas to impact, show the lowest figures. 

Number of Student Equipment Users 

The numbers of users and amount of use of CSIP 
equipment varies widely across projects. In the 
average project, 97 total students per year use the 
equipment, 68 in lab courses, 120 in lecture courses, 
6 each for independent study and student research, 
and 39 for other purposes such as word processing 
(Table 2-4). The average student user employs the 
equipment for a total of 142 hours per year (or 4 
hours per week during two 16-week academic 
semesters). Per user per year, student research 
averages 98 hours, independent study, 96 hours, and 
lab work, 68 hours. 



Table 2-3. Principal investigator's report of the positive 
impact of CSIP project on students by grant 
year: 1985-87 



Area of impacl 


Grant year 


on studenls 


1985 


1986 


1987 



Total 100.0% 100.0% 100.0% 



Skills 


85.6 


75.8 


64.1 


Conceptual or intellectual 








development 


79.8 


67.0 


57.6 


Motivation & attitudes 


83.2 


70.3 


66.3 


Grades 


6.3 


9.9 


7.6 


Attendance 


8.2 


14.3 


16.3 


Increased enrollments 


23.6 


220 


21.7 


Presentations of papers 


56.7 


40.7 


33.7 


Career choices 


42.3 


27.5 


23.9 



Note: Percents add to more than 100 because respondents could 
indicate more than one a r ea of impact 



Source: Assessment of the 1985-87 College Science Instrumentation 
Program. Office o^ Studies and Program Assessment, NSF, 
1990 



Table 2-4. Principal investigator's description of CSIP 
project equipment users, reason for use, and 
amount of use: 1985-87 





Mean 


Mean 


Purpose 


number of 


hours used 




users 


per year 


Faculty users 






All 


4.0 


174.0 


Instruction 


3.5 


109.4 


Research 


2.4 


116.8 


Student users 






All 


97.0 


141.8 


Lab courses 


67.8 


68.0 


Lecture courses 


120.0 


37.2 


Independent study 


6.5 


95 8 


Student research 


6.0 


97.6 


Other purposes 


38.8 


123.2 



Source: Assessment of the 1985-87 College Science Instrumentation 
Program, Office of Studies and Program Assessment, NSF. 
1990. 



Profile of Students Impacted 

A majority of the students impacted by CSIP 
projects were non-minority, non-handicapped, male, 
upper-division science majors. But, as shown in 
Table 2-5, CSIP projects did an impressive job at 
involving measurable numbers of other groups. For 
example: 

■ Of the students involved in the average project, 
42 percent were female, compared to only 13 
percent female in the science and engineering 
labor force"; 

■ Thirteen percent of students involved were 
minority group members, compared to 10 
percent minority in the science and engineering 
labor force; 

» Just over 1 percent of students involved were 
handicapped; and 

■ Of all students involved in projects, 31 percent 
were freshmen and sophomores. 



2 

"National Science Board. Science and Engineering Indicators- 
19S9. Washington. DC: U.S. Government Printing Office. 
1989. (NSB 89-1). Appendix tables 3-2 and 3-4. 



In addition, 6 percent were education majors, and 
60 percent were science majors. 

Some 40 percent of these students will go on to work 
at a related job after graduation, while just under 
30 percent will attend graduate school, about 
17 percent will attend medical school, and about 
6 percent will work as teachers. 

Projects in biology and psychology tended to involve 
a greater than average percentage of females; social 
science projects tended to involve a greater than 
average percentage of education majors; and 
mathematics and psychology projects involved a 
greater than average percentage of nonscience 
majors. Computer science and engineering projects 
were more likely than average to involve students 
who will work in a related job after graduation, and 
were less likely to involve students W'ho will attend 
graduate or medical school, or teach after 
graduation. 

Descriptions from Projects 

PI descriptions of specific impacts of CSIP projects 
on students provide a more concrete view of the 



Table 2-5. Principal investigator's report of profile of students impacted by CSIP project by field: 1985-87 





Field 


Student type 






Chemistry 


Computer 


Engi- 


Earth 




Interdis- 


Psychol- 


Physics/ 


Social 




AJI 


Biology 


Science 


neenng 


Science 


Math 


ciplinary 


ogy 


Astronomy 


Sciences 














[mean) 












In CSIP project 
























Female 


42.3% 


56.2% 


51.0% 


32 7% 


13.6% 


34.1% 


43.9% 


45.3% 


60.7% 


21.9% 


47 7<, t 


Minority 


12.8 


12.7 


14.9 


15.5 


13.3 


3.4 


11.2 


7.1 


7.7 


15.3 


19 7 


Handicapped 


1.3 


1.0 


4 


1.5 


4.2 


.2 


.2 


.9 


1 4 


1.0 


4 0 


Lower division 


31 1 


30.0 


34.9 


223 


22.3 


25.7 


76.4 


31.6 


40.8 


29.9 


22 4 


Education majors 


55 


4.9 


4.5 


5.1 


7.6 


9.1 


10.6 


5.5 


2.2 


4 1 


13.0 


Science majors 


60.2 


62.4 


57.4 


74.1 


68.2 


61.9 


14.3 


58.4 


49.4 


60.2 


63 9 


In department 
























Attend graduate school .... 


28.9 


23.0 


34.3 


13.8 


13.1 


367 


24.2 


32.5 


294 


44.8 


26.2 


Attend medical school 


16.6 


29.2 


25.4 


1.7 


3.0 


1.9 


12.1 


16.4 


8.4 


6.9 


84 


WorK in related job 


40.2 


24.0 


32.9 


85 4 


74.3 


47.9 


40.5 


36.8 


30.0 


35.7 


47 7 


Wort< as a teacher 


6.4 


10.0 


4.7 


2.4 


2.1 


7.0 


15.3 


7.3 


6.4 


6.5 


6.0 



Source- Assessment of the 1985-87 College Science Instrumentation Program. Office of Studies and Program Assessment. NSF. 1990. 
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kinds of student impacts represented in these 
abstract statistics. These range from improvements 
in skills and motivation to enhanced preparation for 
graduate school and jobs in related industries. 

■ Computer-driven simulations and demonstrations 
have really made the course material come alive 
for the students. In addition, we can cover more 
material in a semester because phenomena that 
might take several minutes for me to describe in 
lecture can be easily demonstrated (and 
experienced) in seconds. 

■ {Student} skills have improved with better tools. 
Vie opportunities afforded by the new equipment 
automatically enhance the opportunities for 
intellectual development . . . Motivation and 
attendance are both up. We are over-enrolled with 
a waiting list. Several using the advanced 
equipment have produced very good papers, and 
we have a stronger flow of majors. 

■ Vie greater sophistication of the eweriments that 
students are involved in has led to both a greater 
skill level and a higher level of intellectual 
development. Jlxcre is a spill-over effect. Tlic 
rapidity with which the GC/MS provides useful 
information allows students to do more. TJiis in 
turn leads to further development of skills other 
than in the use of the GC/MS and the increased 
richness of the program leads to greater intellectual 
development. Vie acquisition of the GC/MS was 
pan of a plan for improving the program here 
which has raised student morale and motivation 
to the highest level that we have known. Vie 
number of chemistry majors has increased by 
about 50 percent, and the number going to 
graduate school has approximately doubled. TJiis 
is clear evidence of an effect on career choices. 
Student productivity has increased and this has 
resulted in an increase in the number of smdent 
co-authored papers. Student presentations have 
increased in number and in sophistication. 

■ Since the level of the equipment used in training 
requires greater conceptual ability 1 of the students, 
they are motivated to study and work harder, 
which in mm increases their skills, intellectual 
development and attendance. Smdent grades are 
climbing a bit more since their motivation and 
attitude toward the work load has improved. 
Previously only two papers were sent by students to 
Engineering competition (both winning papers)- 
this year we expect to have to screen papers to get 
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to maximum allowed entries. General audience 
presentations have shown a marked improvement. 
We have noted a small slide-over of students to 
our program from the conventional Engineering 
program. 

■ Students tell us the course has been very valuable 
in helping them find good positions upon 
graduation. One student was hired primarily 
because she was the only candidate for a 
laboratory position who had the computer 
interfacing experience. 

■ Two of our students have gone on to pursue 
graduate work in areas that they first explored here 
with the new equipment. Almost all of our 
students are asking for opportunities to do 
undergraduate research. 

■ {Students have} increased computer/instrument 
skills: several students have presented results of 
t 1 research (using the new instrument) at 
American Chemical Society local section 
symposia. 

■ Following the completion of this project faculty 
have obsen'ed a general increase in the quality of 
work completed by students, level of difficulty of 
projects students engage in, and have received 
positive reports on recent graduates from 
employers. 

■ {Project} has had an even greater effect than 
anticipated . . . for at least four reasons. It has: 
(I) greatly reduced the apprehension with which 
students with little practical experience with 
apparatus approach GC/MS and helped them to 
develop confidence; (2) allowed greater 
sophistication in t ;oice of synthesis projects for 
first semester; (3) allowed students to make much 
greater progress in projects; (4) allowed true 
'hands-on' use by students as they see need. In 
sum smdents are able to do more interesting 
experiments and to carry 9 them further. It has had 
equal impact on student research. Since 
acquisition of the computerized GC/MS the level 
of student accomplishment in research has risen 
sharply. Wliile the GC/MS is not essential to all 
projects, the majority of students use it. Tliey 
provide a critical mass and their accomplishments 
set a standard which other students feel the need 
to meet, and so we now have a higher level of 
expectation and accomplishment in undergraduate 
research. 




Impacts on Faculty 

The importance of faculty impacts should not be 
underestimated, since the morale and motivation of 
the teachers of undergraduate science will have a 
direct bearing on the quality of teaching. The 
impact on faculty members in institutions with CSIP 
projects is clear and positive. Among Pis, 80 percent 
indicated that the CSIP program had benefitted their 
faculty, ranging from 92 percent for 1985 grants to 
73 percent for 1987 grants (Table 2-6). 



Table 2-6. Principal investigator's report of the impact of 
CSIP project on selected areas by grant year: 
1985-87 



Area 


Grant year 




Totai 


1985 


1986 


1987 


Tolal 


100.0% 


1O0.0% 


100.0% 


100 0% 


Improved curriculum 


86.0 


96.6 


87.9 


77.2 


Benefitted faculty 


80.0 


91.8 


78 0 


72.8 


Produced transferable 










products 


26.7 


30.3 


28.6 


22.8 


Found other equipment uses .. 


69.4 


62.7 


77.5 


69 4 



Note. Percents add to more than 100 because respondents could 
indicate more than one impact. 



Source. Assessment of the 1985-87 College Science Instrumentalion 
Program. Office of Studies and Program Assessment NSF. 
1990 



Amount of Faculty Equipment Use 

An average of four faculty members used equipment 
associated with a CSIP project, for an average of 174 
hours each per year (or 5 hours per week for two 16- 
week academic semesters; Table 2-4). Faculty 
members used the equipment for both instruction 
and for research. Their instructional use averaged 
109 hours per year (or 3 hours per week for two 16- 
weck academic semesters), while research use 
averaged 117 hours per year (or 4 hours per week 
for two 16- week academic semesters). 

Descriptions from Projects 

There is clear evidence from narrative statements 
that CSIP grants had strong energizing, revitalizing 
effects on many faculty members of recipient 
departments: 



research projects. Tlxerc is also an important 
contribution to morale. 

■ Benefits to faculty include: lab prep (set-up time) 
ami maintenance less time consuming, availability 
of instrument for research (as benefitted three 
faculty), the success of the proposal and project 
was TWiJ}£acficial to PI (a first-year assistant 
professor in tenure track position at time of grant). 

■ Two faculty members benefitted most by learning 
to use all of the different items of equipment. Our 
enthusiasm about the project proposal, getting the 
award, and putting the project in place has not 
diminished. We wish we had more time to 
develop additional experiments which involve the 
project equipment. 

■ Vie CSIP money drew the attention of our 
administration to the importance of Molecular 
Biolog\ } /Biotechnolog\\ TJiey have contributed 
heretofore unheard of resources to the program (a 
total of about $250,000). Vie equipment has 
driven a remarkable improvement in the 
performance of the four faculty using it. I hate to 
imagine what we would be doing now in the 
absence of the CSIP funds. 

■ As other faculty have obscwed our integration of 
microcomputers into the classroom, they have 
become excited about possibilities for their 
courses. Two of our faculty have received and 
completed grants to prepare software for biological 
exercises. Some of this software is in use at other 
institutions. 

■ We have progressed a great deal in using 
computers both in the lab and in our courses. 
Tlxey have given us a great amount of work and 
satisfaction. We have learned a lot; from 
assembly language to learning how to interface an 
A/O converter. 

■ Faculty* skills have been improved through actual 
instrument use and {through} special courses, 
including the American Chemical Society short 
course on Environmental Analytical Chemistry. 



■ By removing one significant inhibition the 
computerized GC/MS has allowed faculty to 
engage in more ambitious and more interesting 



2-6 



Q "7 



Impacts on Departments, Institutions, and 
Beyond 

There is evidence thai many CSIP projects have had 
substantial and widespread impacts, especially within 
the institution itself. Most Pis said their projects 
had improved the overall curriculum, and more than 
a quarter said they had produced instructional 
products that arc transferrablc for use in other 
institutions (Table 2-6). Almost 70 percent of Pis 
reported they have found uses for their equipment in 
addition to CSIP project activities. 



Dissemination Efforts 

Formal dissemination efforts that would spread 
curricular improvements beyond the host institution 
arc not extensive, although two-thirds of the Pis 
reported that their projects have elements worth 
disseminating. Dissemination activities that have 
occurred so far include the almost 40 percent of Pis 
who have given presentations on their projects, the 
quarter who have shared course materials, and the 
20 percent who have published papers. 



Table 2-7. Principal investigator's report of dissemination 
efforts related to CSIP project: 1985-87 



Dissemination effort 



Projects 



Total . 



Elements worth disseminating 

Received outside inquiries 

Received Information from other protects . 
Held coileglat discussions on project... 

Responded to inquiries on project 

Did presentations on project 

Shared project course materiats 

Pubtished project findings 

Shared equipment with other institutions... 



100.0% 

bS 1 
M.2 
27.0 
57.0 
44.9 
39.0 
25.3 
20.2 
11.2 



Note: Percents add to more than 100 because respondents could 
Indicate more than one dissemination effort. 

Source: Assessment of the 1985-87 Cotlege Science instrumentation 
Program, Office of Studies and Program Assessment. NSF, 
1990. 

Informal dissemination efforts are more common. 
More than half of Pis have held collegial discussions 
on their projects, and about 45 percent have 
responded to inquiries (often about their successful 
proposals; Table 2-7). Small numbers have also 
shared software, given tours, given high school 
workshops, written newsletters, shared with other 
departments, taken field trips, and presented faculty 
minicourses. 



Descriptions from Projects 

Elaborations by Pis on the kinds of improvements 
attributable to their projects and the dissemination 
efforts they have undertaken provide further 
evidence of the impact of the CSIP program. 

■ Our astronomy lab program has become excellent! 
We arc doing things that amaze other institutions' 
faculty. We've given two papers, one at an 
international astronomical union symposium on 
the teaching of astronomy, and one at a regular 
meeting of the American Astronomical Society. 
We have established contacts and developed 
materials for many who arc interested in 
replicating our lab program. 

■ Vie GC/MS is now used in organic lab, 
spectroscopy, instrumental analysis, and 
undergraduate research, A continuing education 
course on GC/MS was taught to staff at the 
Illinois Environmental Protection Agency. We 
have developed a lab on the analysis of 
benzodiozedinc tranquilizers that is being prepared 
for publication. Preliminary results were reported 
at the New Orleans American Chemical Society 
meeting. 

m We vscrc able to advance the curriculum by 
offering a networking lab class, which increases the 
student's learning experience by giving hands-on 
opportunities. We've had indirect impact on the 
rest of our facilities in that the equipment 
purchased for the project showed its capabilities 
and provided the model for the upgrade of the rest 
of our labs. 

m {Products include} an experiment involving 
quantitative analysis by GC/MS {which} has 
been published as a result of this grant in the 
Journal of Chemical Education. Tfus represents 
the first instructional procedure of its kind in print. 

m Because I wanted a forum to share my thoughts 
on this course, problems encountered and solved, 
I organized a state-wide symposium on 
biotechnology, which attracted faculty from most 
of the {state} system (both biolog\> and chemistry 
departments represented). This has now evolved 
into a superb state-wide organization dealing with 
biotechnology. 
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Tlie curricula developed have been shared by 
presentation of papers at the American Chemical 
Society Meeting in 1986, ASBC meeting in 1988, 
publication of a paper in The Journal of 
Chemical Educalioa and {through} response to 
numerous inquiries from individual faculty from 
Delhi, India to Czechoslovakia! 

We have given seven different presentations on our 
work at local meetings (the state Association of 
Physics Teachers), and one at the national 
meeting at Cornell (summer 88), on software and 
hardware developed, and on experiments 
developed. 

A presentation was made at the 1987 New Orleans 
meeting of the American Chemical Society. 



Equipment use and project discussions with five 
local companies and two nearby colleges have 
taken place. Tliree of the companies now have 
purchased equipment identical to ours. 

m Wc believe that this instrumental support makes 
the idea of an integrated laboratory more viable. 
71\is should interest other institutions as should 
some uf our specific materials. Wc have had 
several inquiries from other institutions about the 
project and several requests for copies of the 
proposal. We have presented material at an 
American Chemical Society meeting and will 
present more. Tjxcrc arc as yet no publications, 
but we anticipa.c several We are willing to share 
use of the equipment with local institutions, but 
have no formal arrangements to do so. 
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Chapter 3 

CSIP Impacts Revisited: Site Visit Findings 



Highlights 

Post-survey site visits were made to a stratified 
probability sample of 49 CSIP projects by teams 
consisting of a contractor representative and a peer 
reviewer an academic scientist/engineer who was 
familiar with the CSIP program and knowledgeable 
in the field of the project being visited, The visits 
generated several kinds of information: 

■ Vignettes written by the independent peer 
reviewers who participated in the site visits 
provide concrete illustrations of the many 
different ways that CSIP projects have achieved 
educational impacts far in excess of what might 
have been expected from the modest financial 
investments represented by the initial CSIP 
grants. Some instances were found of projects 
that had not worked out as well as had been 
hoped, but the site visits produced many more 
success stories than stories of promise 
unfulfilled. 

■ Site visit assessments of the accuracy /validity of 
grantee-provided questionnaire data indicate 
that while most questionnaire data were 
accurate, grantees tended to err in the direction 
of understating project impacts. 

■ An important indicator of the extent to which the 
CSIP program was responding to needs that 
otherwise would not be addressed is the 
assessment that 85 percent of the 1985-87 CSIP 
projects would not have been implemented or 
would have been significantly scaled back in 
scope and impact had the grant not been 
received. 

■ Sixty-nine percent of CSIP projects were judged 
to be highly successful in achieving their original 
goals; nearly all of the rest were judged 
moderately successful in this respect. 

■ Most CSIP projects were judged to have had 
positive educational impacts on areas such as: 

Students' understanding of the subjects being 
taught with the project (90%); 

- The quality of teaching in the affected 
courses (88%); and 

- The quality of students' preparation (82%). 



Areas where significant problems were found 
were in ordering, installing and learning to use the 
equipment (42*v ), and in operating and/or 
maintaining the equipment (3S f ,7 ). 

For 84 percent of the projects, the Pi's initiative, 
effort, and time on task were judged to have 
been highly positive factors contributing to the 
project's success in achieving its objectives. 
There were other contributing factors as well for 
many projects, but none as important as the PI. 



Qualitative Findings 

The site visit teams were extremely impressed by 
some of the projects visited. Some examples from 
the peer reviewers' summary assessments follow, 

■ Tfiis project is doing extremely well. Apart from 
its direct impact on several courses and student 
research participation it has had a catalyst effect 
on several other activities in the department. 
Tliree additional CSIP proposals were generated 
(two thus far funded) and other sources of support 
for biology programs located. Together with 
industry donations of equipment and some 
additional funding from the college, biology has 
created an impressive teaching/research 
environment for its majors. Tlier.e is a definite 
upbeat feeling among the faculty, despite hcaxy 
teaching loads created in part by the success of 
their program. They 1 arc also generating an 
extraordinary number of students who continue on 
to some graduate work in the field (20-25 per year) 
for a college with a total enrollment of 1,000. 

■ Tltis is a terrific project! Tficy have leveraged 
N'SF's 15.7 K into a beautifully equipped lab on 
which they have spent almost 80K (over a 4-1 
match). Hxe department head says that "we have 
never had a grant on this campus with this much 
impact." Tlicy have upgraded the curriculum, are 
using the instruments in biology* and anthropology 
as well as chemistry, arc preparing to offer pre-med 
and pre-dent, now offer a concentration in 
chemistry under the physical science degree, and 
all attribute the upsurge to a snowball effect 
created by the CSIP grant. Tliis project is cxactlv 
what CSIP wanted to accomplish; NSF should go 
sec this one and take the congressional committee 
members with them. 
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« {Vic institution} now has a state-of-the-art 
instrument room, which Perkin-Elmer helped 
them design, Vie CSIP equipment is complete, in 
place, and in use in instruction, A permanent 
chemistry position has been added as a result of 
the CSIP instrumentation, and two new courses 
have been added. Tlic state Science and Math 
High School is now at {the institution} (some feel 
as a result on CSIP), a subsequent state grant has 
been received and the college is now helping the 
chemistry department to find additional funding 
sources, Tliis CSIP activity at {the institution} is 
a textbook example of what CSIP was dcsig?icd to 
accomplish. 

• Project is doing very well, PI is older and has 
been at the institution for over 30 years, he says if 
he had been denied CSIP funding he probably 
would have retired. As it is, he gave his 
nights/wcekends/summers to setting up the 
project— which has since become the centerpiece of 
a newly dynamic department. Students benefit 
from the equipment and PI is doing research and 
publishing again. PVs efforts have been 
exceptional, 

m {Vie PI} decided to split the system (both 
hardware and software) into two independent 
systems, one for analysis and one for graphics, 
and bought different hardware (Zenith) and 
software (MAP and MYSTAT) for these purposes. 
Tlic result is a stunning package with unbeatable 
graphics and virtually unlimited analytical 
capability. Vie chemistry department is copying 
his system, and he has helped the biology 
deponment write a CSIP grant, Viis is a genuine 
model. NSF should visit {this school} and 
disseminate this one. 

m Viis project is in place, on track, and very well 
received, Tlic only variation from the proposal is 
a Varian AA instead of Perkin-Elmer (PE did not 
bid under the state requirements). Viis was the 
first chem grant, and started the "ball rolling" (two 
more since thcn h Both the PI and the department 
head say CSIP was directly responsible for 
acquisition of a Mass Spec (58k). Tlic CSIP 
instruments and the MS arc used in recruiting and 
they anticipate 3-4 graduating majors per year 
(compared with past average of two per year). 
Still small numbers, but now they have a means of 
recruitment. Vic PI feels that CSIP has a role in 
her tenure and promotion. This project is seen by 
all as very successful for itself and very influential 
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in stimulating a climate for change on the 
campus. 

m Vie support of the administration, from chair 
through dean through higher administrators has 
been substantial, Vierc has been some release 
granted to the PI (2 teaching hours out of 12), 
although this was not recogiized in his report. A 
substantial overmatch of S35,000 was originally 
generated. Subsequent to that an additional 
$30,000 was also provided. Since the report, an 
additional major piece of equipment was also 
provided. Space in the school is always at a 
premium, and they have used this grant to leverage 
a future lab (by 1992) with over $100,000 in 
renovation. Vie PI received a $2,500 Meritorious 
Pcrfonnancc and Professional Promise Award for 
getting this proposal. It is clear that the proposal 
was initiated by the PI, and that the chair and 
dean then took a major role in leveraging the 
money to maximum advantage. 

m It would be difficult to imagine a project more 
successful than this one: Vie availability of one- 
time state appropriations allowed 3 to I matching. 
All equipment is up and running without glitches. 
All courses in program were reviewed and revised; 
new equipment has been introduced in general 
courses to sophomores(f), and then used in 10 
other courses in department as well as senior 
seminar and master's projects. Formerly they 
showed slides and used data from books; now 
they use labs and field work. Vic system is fully 
computerized-even lab reports are written with the 
wdrd processor and spreadsheet programs. Vie 
program, according to dean, is now a "flagship" of 
the School of Health Sciences; the department has 
been given a new tenure stream faculty position, 
which all uitcn'icws indicated as unusual and the 
clearest sign of institutional support. Vie grant 
was the first such on campus. Vie dean cited the 
program as an important factor in a recent 
$100,000 state grant to interest inner city youth in 
science. Vieir masters program has grown 
dramatically. Vtey boast "100 percent placement" 
of Environmental Health program graduates; every 
E.H. internship leads to an offer, and more 
students onto graduate school. Vic E.H. 
laboratory 1 is now on all campus tours to rccmit 
students. Program has gained prestige on campus 
and with industry; "doors that used to be closed 
arc now open"(PI). In all cases, the PS, the 
department chair, and the dean said it was the one 
CSIP grant that made all of this possible. The 
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university would not have even supported the 
program to such an extent by itself. 

Of course, not all CSIP projects were as 
spectacularly successful as those noted above. For 
one reason or another, a few of the projects visited 
clearly had not been as successful as the PI (and 
NSF) had hoped. 3 

■ One project suffered from a severe fiscal crisis at 
its host institution, a small private college that has 
been experiencing declining enrollments, faculty 
cutbacks and growing financial difficulties in 
recent years. Vie school was unable to provide 
even the minimal matching funds until fully 3 
years after the CSIP award (apparently, an honest 
misunderstanding about the amount of time 
allowable under the program ). Consequently, the 
project has been very slow to develop. Jlie 
equipment has now been assembled, but problems 
of inadequate space, increasingly limited PI time, 
and acute declines in department enrollments 
have conspired to limit its impact. 

■ Another project at a small private college involved 
a single large instrument and was aimed at upper 
division students. Since the original proposal, the 
number of upper-division students in the depart- 
ment has declined to a level of only one or two 
graduates per year. In any case, the distributor 
from whom the instrument was purchased went 
out of business shortly thereafter, and the PI has 
had great difficulty keeping the instrument in 
working order. At the time of the site visit, it was 
down again, and it had been continually out of 
scn'ice for the past 6 months. 

■ TJic PI at this mid-sized public institution was an 
untenured recent PhD in a department whose 
senior members were not equipment oriented. 
Since the award, two other young faculty' members 
who had planned to participate in this CSIP 
project had been denied tenure and had left the 
institution. Without their support, the installation 
and integration of the project equipment has taken 
more time than the PI had anticipated and has 
limited his ability to participate in other tenure- 
enhancing activities. If he, too, is denied tenure, 
he fears the project equipment will fall into disuse. 



Vhe examples include excerpts from presurvey site visits as well 
as from postsurvey visits. 
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■ At another mid-sized public institution, the co-PI 
who was the principal initiator and author of the 
CSIP proposal has since taken on administrative 
and other responsibilities at his institution and has 
turned the project over to the other co-PI, whose 
interests in the equipment arc very narrow. Tlie 
equipment is being used, but less extensively and 
with considerably less student impact than KSF 
had anticipated. 

In a large program such as CSIP, it seems inevitable 
that there will be a few problem projects such as 
these. However, such projects were very much the 
exception, not the rule. Most commonly 
encountered were solid, well-implemented projects 
that are accomplishing their goals, and maybe a little 
more. Additional examples from the peer reviewer 
summaries follow, 

■ Tliis is a project that will produce a modest 
impact each year for a long time, perhaps more 
than 30 years. Vie cumulative result should be 
substantial. Vie instrument, a high quality gravity 
meter, has a very long useful life. It will produce 
meaningful geological information for several 
decades. Already two smdent projects have been 
completed. V:c availability of the instrument has 
made it possible to prepare a proposal to obtain 
funds for more student project work. Student 
laboratory exercises are currently being prepared 
through cooperation between geology and physics 
faculty' members. Vie availability of this gravity 
meter opens ongoing opportunities for sUident 
projects, and the PI is actively pursuing these 
opportunities. Vie gravity meter definitely 
enhances the quality of undergraduate education 
at {this institution}. 

■ Viis project is doing well. Viis is one of the finest 
examples of what the CSIP program is about that 
I've seen. It is the core for a totally integrated 
modem lab program for the entire department. 
Staff arc enthusiastic. Viey also have been 
remarkably successful in using CSIP/NSF funds 
as stepping stones to other funding. 

■ 772/5 is a model program. Vie funds have 
provided equipment that allow students to acquire 
data and analyze data in state of the art fashion. 
The PI and collaborating faculty have done an 
outstanding job of incorporating this equipment 
into their program. Some of the software and 
hardware could be useful to other schools if some 
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method of distribution could be found. Carefully 
conceived, well executed, well documented. 

Tlic project is doing extremely well. All of the 
project goals have been attained. Tlic physics 
department staff have given time and careful 
attention to this project. Vie equipment directly 
impacts approximately 10 students (majors) per 
year. However, indirectly it is impacting the 
laboratory structure of all of the physics classes. 
Lab procedures and experiments arc being 
changed even in the Introductory Lab to better 
prepare the smdents to deal with computer /experi- 
ment interfaces and computer/data acquisition 
techniques. 

Tliis project is quite successful. Hie PI and his 
colleagues have fully implemented the proposed 
modifications to existing course and curricula. 
Tlicy are now using the equipment to move 
bVyond the initial applications. Viere has been 
widespread departmental involvement in 
computerization of laboratory activities and 
considerable positive spin-off in undergraduate 
and graduate student research. 

Hie PI and his colleagues used this proposal to 
leverage more funding from the institution and, 
hence, have , carried this program farther than 
originally planned. 

Tliis project, as well as this entire institution, is 
growing and developing at an incredible rate. Vie 
ICP is being used as planned by the PI in his 
classes. In addition, much student undergraduate 
research is being conducted, as evidenced by a 
number of publications. Instructors in other 
departments have found valuable uses and their 
involvement is leading to more interdisciplinary 
activities. As this institution has, as of now, no 
graduate program in the field, their students must 
go to other institutions for such work; but in a 
number of instances involving several different 
institutions this instrument is not available, and 
they return to this institution for their research, 
Hie PI has been put on their committees to 
expedite the process. 

Tliis grant has had a major positive impact on the 
biology- program. Prior to the grant, the major 
emphasis was on systematic biology, ecology, 
natural resources, etc. T)ie addition of the 
biotechnology has resulted in a new minor in 
cellular and molecular biology despite the fact that 



no new courses were introduced. This came about 
because of the upgrading of courses in genetics, 
immunology, cellular molecular biology, etc. 
Because of this grant they anticipate a new option 
in biotechnology. Vie departmental chair 
indicated that these changes resulted in increased 
medical school admissions. 

■ Tliis is clearly an example of a successful 
operation of the !\'SF program to enhance 
undergraduate education in the sciences. Hie 
requested High Pressure Liquid Chromatograph 
(HPLC) was purchased, installed and used in the 
intended course on schedule and without mishap. 
It has had a positive impact in the intended course 
which is offered at a level affording a good 
undergraduate preparation in analytical chemistry . 
Such a preparation is very important for the typical 
career goals of the student population in this 
department. Since this instrument is only used for 
a part of the Instrumental Analysis course, it could 
not have revolutionized things; but a positive 
impact was evident. Hie instrument is 
furthennorc being used in undergraduate student 
research and figures slightly into a new course on 
microprocessor controlled instrumentation. It may 
also figure into a new biochemistry lab course 
under development. Two faculty use the 
instrument in their direction of student research, 

■ {The PI} has developed an outstanding program 
in experimental psychology. Tlic program allows 
students to experiment on an individual basis by 
using computers. {Tlic PI} first initiative was the 
upgrading of his statistics course. Tliis resulted in 
the development of an entire laboratory manual 
for the Mac Plus and Mac II. Tliis has not been 
published, but it is a perfect example of what 
needs to be shared and made "public" by CSIP 
investigators. Once again, the weakest part of the 
CSIP program shows itself hcre-failure to provide 
for dissemination of information. {The PI} is 
now working on the psychology manual, Tliis, 
too, should be disseminated. 

■ This project is very successful. It has brought an 
expensive, sophisticated, and essential instrument 
to the chemistry department. Without this funding 
they would probably not have this capability. Vie 
instrument is being used by several groups within 
chemistry (organic, physical, and analytical) for 
undergraduate research. Access to this instrument 
has generated much ptide and enthusiasm within 



the department as well as providing a state-of-the- 
art instrument for training undergraduates. 

Vie {project} is a mode! that hus made a positive 
impact at all levels of the college, Vic curriculum 
interfaces with related disciplines and it has 
received full institutional support. Jlie course is a 
product of innovative change within the 
department. TJie laboratory provides a hands-on 
experience in molecular biolog\- that has resulted 
in improved skills, increased numbers of 
independent study projects, enrollment in related 
courses, and student presentations at national and 
regional meetings. Vie course is well designed 
and successfully team-taught. {Vie project) has 
achieved the original goals and bevond the 
expectations of the dean. 

Impact on students appears substantial-excited, 
enthusiastic to work with research-level equipment, 
rather than just being told such equipment exists. 
Anecdotal evidence suggests students now working 
in industry found experience on equipment helpful 
(especially to get job in first place). 

Viis project is doing exactly what the PI proposed. 
It is providing a close to the state-of-the-art Digital 
Signal Processing facility to a well defined 
clientele, the students in the DSP class. It is also 
addressing needs generated by their research and 
allowing analysis of data. Viey have been able to 
build on the original equipment granted and create 
an excellent facility. 

Vie impact of this instrument on upper-level 
students is very high, as the instrument sees 
extensive use in three courses: molecular 
spectroscopy (required for the major), inorganic 
chemistry (required) and methods of measurement 
(optional). Each student uses the instrument for 
one to two afternoons per semester; the rotating 
schedule of labs indicates that the instrument is 
used extensively nearly every week of the semester. 
Vic use of this instrument in the introductory 
organic course has not yet been developed. Viis 
instrument is also heavily used by students 
involved in independent research. Vie response of 
students has been uniformly positive and 
enthusiastic. 

This instrument was useful for catalyzing the 
conversion of a stockroom into an instrument 
room housing not only the FTIR, but also four 
other instruments. Vie instrument is maintained 



by faculty, in conjunction with the electronic shop 
staff Such a task, like the task of writing the 
research grants for shared instrumentation, is 
viewed by the department chairman as a standard 
part of the job of faculty. Vie Dean of Faculty, 
however, commented that equipment maintenance 
is "a real problem" and suggested that it is a poor 
use of faculty resources which may now not be 
appropriate. 

Validation Findings 

After inspecting the project equipment, curricula, 
publications, and other materials, and interviewing 
the PI, the department head, the dean, and others, 
the post-survey site visit team members jointly 
assessed whether the Pi's questionnaire understated, 
overstated, or accurately communicated the projects 
actual status with respect to several dimensions. The 
findings, weighted to represent all 1985-87 CSIP 
grantees, are summarized in Table 3-1. 



Table 3-1. Site visit assessment of the accuracy of CSIP 
grantee questionnaire data: 1985-87 



Dimension 


Assessment of questionnaire 
information 


Total 


Under- 
states 


Over- 
states 


Accu- 
rate 


Extent o! progress in 










implementing project. 


100.0% 


16.9% 


12.2% 


70.9% 


Extent of Change from 










planned equipment 


1000 


10.9 


0.0 


89.1 


Extent of change from 










planned objectives 


100.0 


6.0 


00 


94.0 


Extent of 










resource leveraging. 










Total 


100.0 


24.6 


7.5 


67.9 


Prefects reporting 










$5.000+ 


100.0 


45 7 


8.7 


45.6 


Projects reporting 










<$5.000 


100.0 


18.5 


7.1 


74.4 


Extent of other 










project Impacts 


100.0 


35.8 


0.0 


64.2 


Type and extent 










of dissemination 


100.0 


5.7 


0.0 


94.3 



Source: Assessment of the 1985-87 College Science Instrumentaiton 
Program, Office of Studies and Program Assessment, NSF, 
1990. 



Generally, the site visitors found the questionnaire 
data to be quite accurate. The only clear pattern 
that emerged was a strong tendency of Pis to 
understate positive project impacts when reporting 
on their questionnaires, a finding that ran counter to 
expectations. For instance, 36 percent of Pis were 
judged to have understated the educational impacts 
of their projects, and 46 percent of the Pis who 
reported $5,000 or more in equipment or resources 
over the required match were found to have 
significantly underreported these dollar amounts. 
Exaggeration of outcomes was almost never a 
problem. This is an important finding, which 
suggests greater effort will be needed in future 
evaluations to obtain a full and complete accounting 
of project impacts, both financial and educational. 



Other Site Visit Findings 

Would Projects Have Occurred Without 
CSIP? 

Given that ail of the supported projects were judged 
by CSIP proposal reviewers to be highly meritorious, 
the possibility arises that many might have come to 
fruition even if the CSIP program had not existed. 
In some cases, for example, it is possible that Pis 
may have had explicit commitments for full project 
support from other sources in the event that the 
CSIP award was not received. The hypothetical 
question of what would have happened to the project 
if CSIP funding had been denied was explored in the 
site visit interviews with Pis and department heads. 
The results are summarized in Figure 3-1. 



Figure 3-1: Site visit assessment of whether projects would have occurred without CSIP 




Project would have been mostly or 
fully implemented with institution or 
other funds 



^ A scaled down version of the project 
would have been implemented 



The project would not have been 
implemented 



Percent of Grantees 



45 



Based on the assessments of ihe site visil teams, only 
15 percent of CSIP projects would have been mostly 
or fully implemented with funding from institution 
or other sources if the CSIP grant had not been 
received. For the overwhelming majority of projects 
(85%), the CSIP grant was of critical importance to 
the development of the project; without ii, projects 
would either have been significantly scaled back 
(38%) or not implemented at all (47%). Perhaps 
more than any other single finding from this 
evaluation, this assessment underlines the extent to 
which the CSIP program is addressing real needs 
that would not otherwise be met. Without CSIP* 
many of the proposals to obtain modern 
instrumentation for use in undergraduate 
science/engineering education, even those most 
needed and most meritorious, would find it difficult 
or impossible to obtain the necessary financing. 



Degrees and Areas of Project Success 

The site \isit teams made summary assessments of 
'the extent to which individual projects have been 
successful thus far in various respects (Table 3-2). 
Most projects (699c) were judged to have been 
highly successful in achieving the goals set forth in 
their proposals, and most of the rest (299c of the 
total) were judged moderately successful in this 
respect. In terms of the four CSIP program goals 
that respondents were asked to consider, upwards of 
80 percent of the projects that sought to achieve a 
particular goal were judged to have been at least 
moderately successful to date. An exception is the 
CSIP goal of establishing equipment sharing 
consortia, where the assessment data arc somewhat 
muddy: rrfany projects were rated as having had no 
success in this area, but most had not had this as a 
goal. 4 



As noted in Chapter 2. only 6 percent of CSIP grantees' 
proposals addressed this goal. 



Table 3-2. Site visit assessment of CSIP project success: 1985-87 



Project success 



Dimension 


Total 


High 


Moderate 


None 


Not ascertained; 
Not applicable* 




100.0% 


69.4% 


29.2% 


1.4% 


(0.0%) 


Introducing modern instruments to improve the experiences ot 
undergraduate students In laboratories and field work 


100.0 


72.6 


202 


7 1 


(4.2) 


Interfacing computers with scientific instrumentation for 


100.0 


51 .0 


32 4 


16.7 


(30.2) 


Developing new ways of using instrumentation to extend 


1000 


38.0 


51.0 


10.9 


(17.4) 


Establishing equipment shanng capability via consortia or centers 


100.0 


5.4 


7.9 


866 


(37.4) 


Leveraging additional funds (beyond the required match) 


100.0 


27.0 


40.2 


32.8 


(0.0) 



•Projects for which assessments were not ascertained or not applicable were excluded when computing other percentages 

Source: Assessment ot the 1985-87 College Science Instrumentation Program. Office ot Studies and Program Assessment. NSF. 1990 
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Another goal that was seldom listed as such in CSIP 
proposals was that of obtaining additional funds to 
expand beyond the scope of the initially proposed 
project. As shown in Table 3-2, over two-thirds of 
all CSIP projects had achieved some expansion 
beyond what was originally proposed, and over one- 
fourth (279c) were judged to have been highly 
successful in this respect. 



Types of Project Impacts and Problems 

The site visit teams assessed whether projects had as 
yet had positive (or perhaps negative) impacts 
(Table 3-3). As hoped, the highest levels of 
consistently positive impacts were found for students 
and for curriculum and instruction (e.g., in the area 
of curriculum, 88% of projects were judged to have 
had positive impacts on the quality of teaching in the 
affected courses). Negative impacts were found in 
only two areas: 

■ On the PI, whose workload often increased 
materially (44%) and whose ability to take on 
other professional obligations was sometimes 
diminished (21%); and 

■ On short-term funding, either because of 
difficulties in obtaining the required matching 
funds (10%), or because once special efforts had 
been made to provide the matching funds, the PI 
or department moved to the end of the line in 
laying claim to any additional funds that might 
become available (7-10% of the time). 

In both of these areas, however, positive impacts 
were also seen. Obtaining CSIP grants was thought 
to have had positive impacts on future funding 
prospects for the PI or department far more often 
than negative impacts (62-65% vs. 7-10%), and 
CSIP-rclatcd PI workload increases were sometimes 
accompanied by salary increases (29%) or improved 
chances of obtaining tenure (45% of Pis who had 
not been tenured at the time of the award were 
viewed by site visitors as strong candidates for 
tenure after the award). 



Table 3-3. Site visit assessment of CSIP project impacts 
1985-87 





Nature of impacl 


Type oi ;mpact 




Positive 


NO 


Negative 




Total 








impac! 


impact 


imp?.?' 



Curriculum and instruction 

New/modified courses 100.0% 73 0% 27.0% 0.0°/, 
New/upgraded topics 
Teaching of subject.... 

Department faculty 



Morale 

Interest in upgrading . 
Recruitment 

Principal Investigator 

Possibility for tenure.... 

Salary 

work load 

Other professional 
obligations 

Students 

Level of interest 

Understanding 

of subject 

Preparation 

Science instruction 

Usefulness as model . 

Funding 

Acquiring matching 
funds 

Leveraging additional 
money 

Department 

Prestige 

Funding level 

Space 

Student enrollment 

Student recruitment.... 

Faculty recruitment 

instructional program 

Institution 



Grant administration 

process 100 0 40.2 58 3 1 5 

Institutional support 100 0 68.5 30 1 1 4 

Prestige 1 00.0 58.8 41 .2 0.0 

Student recruitment 100.0 41.7 58.3 0.0 

Source: Assessment of the ^85-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment, NSF. 
1990. 



100.0 


87.8 


12.2 


00 


1000 


87.6 


12.4 


0.0 


100 0 


71.3 


27.2 


1 .5 


100.0 


74 7 


23.8 


1 4 


1000 


48.8 


51.2 


00 


100.0 


45.0 


55.0 


00 


100.0 


29 4 


70.6 


O.C 


100.0 


11.0 


44.9 


44.1 


100.0 


29.8 


49 4 


20 7 


100.0 


76.2 


23.8 


0.0 


1000 


90.5 


9.5 


00 


100.0 


82 1 


17.9 


0.0 


100 0 


57 6 


40.9 


l 5 


1000 


83 2 


68 


10.0 


100.0 


648 


28 4 


68 


100.0 


87.6 


11.5 


1.5 


1000 


61.6 


28.2 


10.2 


100.0 


38.5 


59.6 


1.8 


1000 


34.2 


65.8 


00 


100.0 


44.8 


55.2 


00 


100.0 


41.1 


589 


00 


1000 


908 


9.2 


0.0 
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The site visit teams also explored the extent to which 
projects had been held up or Pis had to expen 
more effort than anticipated because of certain 
problems (Table 3-4). Significant delays in acquiring 
the matching funds were a major problem for an 
estimated 12 percent of the projects, but 71 percent 
of the grantees had no problems in this area. A 
significant proportion of the grantees (42%) 
experienced significant delays or problems in 
ordering, installing, or learning to use their CSIP 
equipment, and 25 percent had major delays or 
problems. Even so, 58 percent of the grantees 
reported no problems in this area. Problems 
operating or maintaining the equipment were 
seldom major (89c), but neither were they 
uncommon: 3S percent of the projects had at least 
minor problems in this area. 



Factors Contributing to Project Success 

Many factors contributed to the success of individual 
CSIP projects. Other faculty, department and 
institution administrators. and even student 
assistants often played significant roles (Table 3-5). 
By far the most important factor, however, was the 
initiative, effort, and time on the task of the grant PI, 
who was the major contributor to the overall success 
of the project over 80 percent of the time. 

Table 3-4. Site visit assessment of problems contributing to 
CSIP project delays or requiring the PI to expend 
more effort than anticipated to stay on schedule 
1985-87 



Problem area 


Extent of problem 


Total 


Major 


Minor 


None 


Acquinng funds 


100 0% 


11.8% 


17.0% 


71 1% 


Ordering, installing. 










and learning lo use 










equipment. 


1000 


25 4 


16 3 


582 


Operating and/or 










maintaining 










equipment 


1000 


8.5 


29 9 


61.6 



Source Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF, 
1990 



Table 3-5. Site visit assessment of the contributors to the success of the CSIP project: 1985-87 



Contributor to project success 


Assessed contribution 


Total 


Major 


Modest 


Slight 


None 


Negative 


Pi's initiative, effort, time on task 


100.0% 


83.7% 


5.9% 


7.0% 


0.0% 


3.5% 


Other faculty 


100.0 


24.0 


20 1 


27.7 


28.2 


00 


Department administrators 


1000 


30.1 


9.1 


30.3 


25 0 


54 


InsMution administrators 


100.0 


19.2 


24.2 


23.2 


24.1 


9.3 


Student assistants 


1000 


.4 


20.2 


18.5 


59.1 


1.8 


Others 


100.0 


28.1 


16.8 


00 


55.1 


00 



Source: Assessment of the 1 985-87 College Science Instrumentation Program. Office of Studies and Program Assessment. NSF. 1 990 
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A more detailed assessment of the institution's role 
in the development and implementation of CSIP 
projects is presented in Table 3-6. As shown, while 
institutions were often supportive in encouraging Pis 
to submit CSIP proposals and later in providing 
needed supplies and help with equipment 
maintenance and repair, institutions seldom were 
actively invoked in the preparation of proposals 
(19%), and only infrequently gave Pis release time 
to work on the implementation of their projects once 
the CSIP grant was received (13%). 

These findings reinforce the questionnaire data in 
suggesting that, while CSIP grants are made to 
departments and institutions, it is the individual PI 
who is the principal and most important contributor 
to outcomes at each step, from the initiation and 
development of the proposal to the eventual 
installation and actualization of the project itself. 



Table 3-6. Site visit assessment of the institution's role 

CSIP project development and implementation- 
1985-87 



Areas of 
institution 
support 


Level of support 


Total 


Some 
support 
given 


No 
support 
given 


Problem 
area 


AI the proposal stage 










Initiation of 










proposal 


1 00 0% 


42 7% 


57.3% 


0 0% 


Encouragement of 










submission 


100.0 


85.6 


14.4 


0.0 


Preparation of 










proposal 


100.0 


18.9 


81.1 


0.0 


AI the implementation stage 








Release time 


1000 


126 


80.8 


6.5 


Additional space 


1000 


40.9 


54.9 


4.2 


Supplies 


100.0 


76.5 


22.1 


1 4 


Technical support.. .. 


1000 


48.5 


42.5 


9.0 


Provisions for 










maintenance and 










repair 


100 0 


71.9 


6.5 


19.6 



Source. Assessment of the 1S3L>-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990 
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Chapter 4 

Outcomes for Unsuccessful CSIP applicants 



Highlights 

Almost 70 percent of the proposed Pis were denied 
funding for iheir CSIP projects from 1<>85 lo 1987. 
These unsuccessful CSIP applicants report mixed 
effects of the proposal writing experience. On the 
one hand, the planning process was often viewed as 
a valuable opportunity to evaluate current curricula 
and facilities and to establish goals. These plans, 
once set in motion, often came to fruition with 
further effort. On the other hand, more than 35 
percent of these unsuccessful applicants have not 
acquired any of the proposed instruments, often lo 
the detriment of undergraduate science education at 
their institutions. 



Outcomes 

In stimulating Pis to conceptualize, plan, and budget 
projects to benefit undergraduate instruction at their 
institutions, and then to convince their departments, 
institutions, or others to commit matching funds in 
the event a grant is made, the CSIP program may- 
create a momentum that will continue even if the 
grant is not awarded. 

Information relevant to this speculation is presented 
in Table 4-1. As shown, a majority of the 
unsuccessful CSIP applicants did continue their 
efforts to obtain project funding support (61%), and 
of these, 76 percent have been able to obtain at least 
some such support. Of the 1,681 unsuccessful CSIP 
applicants, 64 percent have been able to obtain at 
least some of the equipment they requested for their 
proposed CSIP project. The mean is 53 percent of 
the original project budget. 

If we assume the average budget for both 
unsuccessful and successful CSIP applicants was 
about the same (i.e., about $47,000), this 53 percent 
of the budgets of 64 percent of the unsuccessful 
applicants amounts to about $27 million of 
additional instructional equipment indirectly 
generated by CSIP. If this is included in calculating 
the return on NSFs financial investment, the return 
increases from 450 percent in direct return (from 
Chapter 1) to about 600 percent return altogether. 



Table 4-1 Proposed principal investigator's report of the 

consequence of unsuccessful application for 

CSIP funding for the implementation of the 
proposed project: 1985-87 



Index 



Percent 



Effort made to oblam alternate funding 
All nonfunded projects 



Alternate funding obtained 
All nonfunded projects. . . 
Those who made effort. . .. 

Proposed instruments obtained 
Those who made effort 



Portion of original CSIP budget 
request obtained instruments represent 

Those who made effort 



Alternate funding sources 

Department 

College/university 

Federal government .... 

Business/industry 

Other 



Further funding efforts anticipated 
Ail nontunded projects 



61 



47 
76 



64 



53 



46 
60 
17 
16 

22 



44 



Source Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment, NSF. 
1990. 



In their questionnaire, unsuccessful applicants were 
also asked to describe in their own words the 
positive or negative effects of their participation in 
the CSIP program. Half reported positive effects of 
one kind or another, including grant writing practice, 
use of the same proposal to submit elsewhere, 
forced examination of the curriculum or department, 
and attention brought to the department. Some 
excerpts from their statements follow. 

■ Kept me alert to new developments and made us 
evaluate the people who are now in the field. 

■ / was able to use basically the same proposal to 
apply to other sources! 

■ After preparation of the first CSIP proposal, others 
were easier. 



Plans were crystallized, goals set. 
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■ Clarified our needs. 

• Communicated effectively to local administration 
need for instrumentation in the chemistry 
curriculum. Focused dialog in the department on 
improving instruction by changing methodology. 

■ Vie door was opened to seek alternative means of 
establishing our goals. 

■ / learned how to prepare an NSF type grant. 

■ // helped me review our program and see where it 
could be improved. 

■ Yes, the attitude of faculty and students improved 
for having been involved or aware of the pending 
proposal. 

9 Provided me with the experience for proposal 
writing. 

■ We were able to revise the proposal and obtain 
enough of the equipment to serve the original 
purpose. 

■ Fart of the narrative was used to solicit funds from 
sources on this campus. 

■ Tlie proposal sened as a catalyst for curricular 
review within the biology department. Extensive 
changes in the core courses and several upper- 
division courses have occurred over the past 2 
years. 

■ Tiie proposal helped initiate evaluating our 
laboratory equipment needs. We have continued 
to pursue and refine the stated equipment needs in 
later years. 

■ Administration recognized at a new level the need 
for faculty support in this area. 

■ Cooperative effort between my animal science 
department and biology department. 

Some examples of negative impacts include students' 
loss of experience with equipment, a perceived 
diminishing of the prestige of the PI or the 
department, and disappointment or discouragement 
on the part of PI or department. Their statements 
include the following observations: 



■ Mild depression on the part of the author. Many 
hours of work produced not even any real evidence 
that the project was seriously considered. 

■ It did delay our 'hands-on 1 introduction of S'MR 
spectroscopy to our students. 

■ Possibly lowered morale after two consecutive 
failures. 

■ We had to wait 4 or 5 years before we could get 
any decent equipment. 

■ It has inhibited expansion of our teaching 
activities in this area, 

■ Caused diversion of funds from one project to 
another and delayed the implementation of other 
projects that must now gather dust on the shelves, 
that are equally meritorious. Hie people of this 
area will lag behind the population centers in their 
knowledge of science, etc. 

■ Frustration— it seems those who really are in dire 
need of external support are not funded vshercas 
those with existing facilities always get more. 

■ // discouraged future proposals-since in this 
proposal the reviews were not negative and there 
was no indication of what could be done to 
improve chances next time. 

■ Delayed by 3 years the establishment of the 
computer teaching laboratory'. 

m Since I could not obtain adequate funding to have 
the equipment necessary to teach experimental 
physics at a level necessary 1 to allow students to 
survive after obtaining a BS, I became 
disillusioned, quit teaching, and went into 
industry. 

■ Lack of funding resulted, indirectly \ in the program 
being dropped. A number of conservative 
individuals resent this type of innovative program. 
Had it been funded we could have overridden their 
votes and influence. Tlie students of course are 
the big loser as the program is no longer available 
to them. Some of the younger faculty also miss 
the advantage of this type program. 




■ Of course our students do not have the benefit and 
training on up-to-date equipment. 

■ It was a significant factor in my decision to leave 
{school}. Several faculty and students who 
realized physiological psychology* had been long 
neglected at {school} were disappointed. Vie 
college experienced some interruption in the 
transition from myself to my replacement. 

■ Was denied promotion and tenure at application 
institution. 

■ It does make it harder to interest and maintain 
faculty when equipment begins to age and support 
is not forthcoming. 

■ Sure discouraged us as a small college. 

■ / could have made teaching more exciting! 

■ Jlxis school docs not have a good academic 
reputation. Your denial of the request just simply 
confirms it. 

It seems that failure to receive funding can have very- 
different impacts. As shown above, specific 
outcomes for individuals and departments are often 
dependent on circumstances unique to the 
submitting institution and individual. 



PART 2 

PROGRAM CHARACTERISTICS 



Chapters 



CSIP Program Reach 



Highlights 

This chapter provides a broad description of CSIP 
program participation during its first 3 years of 
operation. 

Both the number of CSIP awards and the success 
rate of proposals increased between grant years 1985 
and 1987 as increased funding became available. 
During this period, CSIP grants were distributed 
across fields in proportion to the number of 
proposals received. Pis' academic credentials and 
department characteristics generally did not appear 
to affect likelihood of award, and grantees' proposals 
were no different from unsuccessful applicants' 
proposals in their initiation or authorship. However, 
the following factors were associated with likelihood 
of success in winning CSIP awards: 

■ The extent to which the proposal addressed the 
stated CSIP program goals, and especially the 
extent to which it proposed innovative uses of 
instrumentation for undergraduate instruction; 

■ The persistence of individual Pis in submitting 
revised proposals if not successful initially; 

■ (In certain fields) the total number of 
undergraduates being served by the department; 
and 

■ At the school level, the total number of CSIP 
proposals submitted over the 3 year period, 
especially whether the number was one or more 
than one. 



Numbers of Proposals and Principal 
Investigators 

According to NSF program records, 3,226 CSIP 
proposals were submitted d uring the 3 years of the 
program, and 780 grants were awarded (Table 5-1). 
Following an initial surge of 1,335 proposals in 1985, 
the number of proposals levelled off to 922 in 1986 
and 969 in 1987. The trend in grants awarded, 
however, went in the opposite direction: as greater 
CSIP funding became available, the number of 
grants increased from just over 200 in 1985 and 1986 
to over 300 in 1987, Consequently, the program 
award rate grew from only 18 percent in 1985 to 
35 percent in 1987. 



Table 5-1 . CSIP proposals, awards and applicants by grant 
year. 1985-87 



Number of 
and Pis 


Grant year 


Total 


1985 


1986 


1987 


Proposals 










Proposals received . 


3.226 


1.335 


922 


969 


Grants awarded . .. 


780 


234 


?11 


335 


Award rate 


24% 


18% 


23% 


35% 


Pis (undupiicated) 










Total 


2.449 


1.196 


529 


724 


Grantees 


766 


234 


210 


324 


Unsuccessful 










applicants 


1,681 


962 


319 


400 



Source. Assessment of tne 1 985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990 



Some Pis submitted multiple CSIP proposals during 
1985-87 (though never more than one per year). As 
shown in Table 5-1, the 3,226 proposals received in 
1985-87 were submitted by a total of only 2,449 
different Pis; this means that participating Pis 
submitted an average (mean) of 1.3 proposals each 
during this period/* This duplication is eliminated in 
the lower section of Table 5-1 and in all subsequent 
tables in this report, where individual Pis are 
assigned to a single status category based on their 
earliest winning proposal. Thus, a PI who submitted 
an unsuccessful proposal in 1985 and a successful 
one in 1986 is classified as a 1986 grantee; a PI who 
submitted unsuccessful proposals in both 1985 and 
1986 is classified as a 1985 unsuccessful applicant. 
Pis who submitted at least one unsuccessful CSIP 
proposal in 1985-87 and had no winning proposals 
during that period will be classified as unsuccessful 
applicants in this report. 



probably understates (he real rate of resubmission, since it 
is based on computer matching of PI names. Situations where 
a proposal was resubmitted under a different name are not 
included. 



Fields, Levels, and Objectives of Proposed Projects 

As shown in Tabic 5-2, chemistry (21 c /c) and biology 
(219c) received the largest numbers of CSIP grant: 
in 19S5-S7. followed by engineering (1292) and 
physics/astronomy (12%). The distribution across 
fields was very similar for grantees and unsuccessful 
applicants, indicating that CSIP grants in a given 
field were made in close proportion to the number 
of proposals received in the field. 

Grantee proposals were about evenly divided 
between those that would affect upper-division 
students only (45%) and those that would affect both 
upper- and lower-division students (5091). Few 
successful proposals were addressed only to lower- 
division students (5%). Level information was not 
available for unsuccessful applicants from the 1985 
CSIP, but for the 1986 and 1987 unsuccessful 
applicants, proposals had approximately the same 
distribution by level as found for the grantees. 

Grantees and unsuccessful applicants were not so 
similar when asked which of the four CSIP program 
goals their proposed projects addressed (Figure 5-1). 
While over 90 percent tf both groups said their 
projects would promote the first goal, introduction of 
modem instruments to improve the experiences of 
undergraduate students, substantially more gra itees 



Table 5-2. Field and level of proposed CSIP projects by 
award status. 1985-87 



Field and level 


Grantee 


Unsuccessful 
applicant 




Number 


Percent 


Number 


^--cent 



Field 










Total 


752 


100.0 


1.681 


100.0 


Biology 


158 


20 7 


341 


20 3 


Chemistry 


204 


OC D 

£U o 


404 


24 0 


Computer science ... 


32 


4.2 


II/ 


7 n 


Engineering 




IOC 


01 R 

c. I O 


13 0 


Earth science 


37 


4.9 


78 


4.6 


Interdisciplinary 


65 


8.5 


152 


9 0 


Mathematics 


11 


1 4 


33 


2.0 


Psychology 


46 


6.0 


88 


5.2 


Physics/astronomy. ... 


89 


11.7 


213 


12.7 


Social sciences 


24 


3.1 


38 


2.3 


Level 






Upper division only . 


343 


45 1 


282 


39 2 


Both divisions ... 


377 


49.6 


365 


50.8 


Lower division only . . 


40 


5.3 


72 


10.0 


Unspecified 


2 


n.a • 


962 


n.a • 



•Row was not included in calculation of percents 



Source Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990. 



Figure 5-1: CSIP program goals addressed by grantee and applicant proposals: 1985-87 
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than unsuccessful applicants claimed that their 
projects would also involve interfacing of computers 
with scientific instrumentation and other appropriate 
uses of current technology in science and engineering 
instruction (58% vs. 40%), or development of new 
ways of using instrumentation to extend instructional 
capabilities (59% vs. 41%). It appears that CSIP 
proposal raters looked for projects that involved 
more than just making additional equipment 
available to undergraduates; projects that also 
involved creative ways of using equipment to 
enhance curricula appear to have been more likely 
than others to receive funding. 

We might also speculate that successful CSIP 
proposers had a better understanding than 
unsuccessful applicants of the importance of 
addressing NSFs specific objectives for the CSIP 
program, as stated in the guidelines for proposals. 
Proposals that overlooked these program objectives 
apparently had lower chances of success in obtaining 
funding. 



Characteristics of Grantees 
and Unsuccessful Applicants 

Most CSIP grantees (93%) hold doctorate degrees 
(Table 5-3). About three-fifths were tenured and 
were associate or full professors when the proposal 
was written, and about three-quarters of them are 
now ~ 2 to 4 years later. At the time of the survey, 
grantees had been at their current institutions for an 
average (mean) of 12 years and had held their 
highest degree for an average of 15 years. The mean 
teaching load among grantees was 12 hours per 
week, both when the proposal was written and 
currently. 



Table 5-3. Academic characteristics of CSIP proposed 
principal investigators by award status: 1985-87 





Grantee 


Unsuccessful 
applicant 


PI characteristic 


At time 




At lime 






of 


Currently 


of 


Currently 




proposal 




proposal 





Tolal 


00.0% 


100.0% 


1 00.0 % 


100.0 


Highest deqree 










Doctorate 


n.a * 


93.2 


n.a 


90 1 


Master's 


n.a 


6.0 


n.a 


B.7 


Bachelor's 


n a 


.8 


n.a 


1.2 


Tenure status 










Tenured 


58.4 


74.2 


62.9 


74.8 


Untenured 


38.5 


22.6 


33.0 


20.2 


Not tenure track 


3.1 


3.2 


4.0 


5.0 


Academic title 










Full professor 


37.0 


43.7 


41 6 


47.5 


Associate professor . 


26.3 


30.8 


22.0 


27.7 


Assistant professor 


33 8 


20.8 


25.3 


11.6 


Department chair. 


.8 


3.0 


4.1 


34 


Other 


2.1 


1.7 


7.0 


9.8 


Teaching load 










Mean hours per week 


12.0 


12.0 


12.8 


126 


Years since ftiqhest deqree 










Me^n 


n.a. 


15.2 


n.a. 


17.9 


Years on current faculty 










Mean 


n.a. 


12.4 


n.a. 


13.9 



•Information was not obtained 

Source: Assessment of the 1985-87 College Science instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990. 



Unsuccessful applicants as a group have a very 
similar demographic profile; if anything, they as a 
group were slightly older and slightly more senior 
academically than the grantees. This pronounced 
similarity between the two groups, both before and 
after the CSIP proposal/award process, suggests 
that the Pi's academic credentials were not a major 
factor affecting CSIP awards. These data do not 
allow a clear conclusion with regard to the effects of 
award receipt or nonreceipt on the Pis 1 subsequent 
career development, at least not in terms of such 
elements as tenure, title, and teaching load. Even 
so, qualitative data from site visits presented in 
Chapter 3 suggest there may be some positive effects 
for awarded Pis. 



Development of the Proposal 

Funded CSIP proposals seldom originated as a 
result of initiative from the college administration or 
from the department head (Table 5-4). Most 
grantees' proposals were initiated by the PI (78%), 
and most also were written by the PI (83%). It 
would have been interesting if unsuccessful 
applicants had evidenced a different pattern, but 
they did not: most nonfunded proposals were also 
initiated and written by the proposed PI. 




Table 5-4. Factors in the development of CSIP proposals 
by award status: 1985-87 







1 Inn ifs^ACcfi il 

unsuccessful 


Proposal factor 


Grantee 


applicant 




Drrtrv>cal initiator 






PI 


78 4% 


79.6% 


CoMeQe administration 


1 3.0 


8 9 


Department head 


2.6 


6.7 


Other 


6.0 


4.8 


Proposal author 






PI 


83 4 


85.6 


Department committee 


12.2 


11.7 


Other 


4 4 


2 7 


Previously applied for funds 






for this or similar project 






Total (1985-87) 


34.5 


14.4 


1985 


14.9 


10.7 


1 986 


44.0 


24.0 


1 987 


42 4 


15.7 


Previously received 






other qrant fundino 






Federal and /or other 


71.5 


65 6 


Federal 


48.9 


39 8 


Other 


53.6 


47.2 



Source Assessment of the 1985-87 College Science Instrumentation 
Program, Office of Studies and Program Assessment, NSF, 
1990. 



One factor that did differentiate grantees from 
unsuccessful applicants was whether or not the PI 
had previously applied for funds for a project that 
was the same as or closely related to the current 
CSIP proposal. Grantees more often reported such 
previous experience, especially in 1986 and 1987 
when more than 40 percent of grantees reported 
previous applications, primarily unsuccessful CSIP 
applications. In contrast, only 16 to 24 percent of 
the unsuccessful applicants applied previously. This 
suggests that, after the first year of the program, 
many of the successful CSIP proposals were 
submitted by Pis who had been turned down on 
their first try and successfully addressed the 
reviewers' concerns in their resubmittals. 

Another indication of the beneficial effects of 
practice and experience in writing proposals is that, 
in addition to previous applications for their CSIP 
projects, grantees were more likely than unsuccessful 
applicants to report previous experience in obtaining 
grant support for other projects (72% vs. 66%), 
from both Federal and non-Federal sources. 



Characteristics of Pis 1 Departments 

In terms of mean number of faculty, percent of 
faculty holding Ph.D. degrees, or mean number of 
graduating undergraduate majors, grantees' and 
unsuccessful applicants' departments were quite 
similar (Table 5-5). 



Table 5-5. Characteristics of departments submitting CSIP 
applications by award status: 1985-87 



Department 




Unsuccessful 


Grantee 




characteristic 


applicant 


faculty 






Mean number 


10.3 


11.2 


Percent with Pn.D 


77.5 


73.7 


Graduatinq maiors 






Mean number per year 


24 5 


23 1 


Undergraduate fall enroilees 






Mean number, all S/E fields 


512 


360 


Biology 


454 


228 


Chemistry 


473 


281 


Computer science 


651 


841 


Engineenng 


266 


287 


Earth sciences 


202 


203 


Interdisciplinary 


461 


204 


Mathematics 


1,140 


711 


Psychology 


1,221 


881 


Physics & astronomy 


342 


270 


Social sciences 


1.557 


859 



Source 1 Assessment of the 1985-87 College Science Instrumentation 
Program, Office of Studies and Program Assessment. NSF. 
1990 

However, in terms of total departmental 
undergraduate opening fall enrollment in 1987, the 
mean for grantees was substantially higher than that 
for unsuccessful applicants (512 vs. 360). This 
difference was seen in all fields except computer 
science, engineering, and earth science, and was very 
pronounced in mathematics, psychology, and the 
social sciences, where grantees' departments served 
averages of 1,140 to 1,557 undergraduates versus 
averages of 711 to 881 for unsuccessful applicants' 
departments. These fields all serve relatively large 
numbers of lower-division students, and one might 
speculate that CSIP projects that incorporate 
modern instrumentation into the curricula of lower 
division students would be more likely to receive 
funding if the number of students to be affected is 
relatively large. 
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Institution Success in Obtaining CSIP Awards 

The institutions eligible to participate in CSIP during 
the 1985-87 period were the country's 2,706 4-year, 
non-doctorate-granting schools. A total of 811 
schools, 30 percent of those eligible, sent one or 
more CSIP proposals to NSF at some time during 
those 3 years (Table 5-6). This total is not the sum 
of applicant schools for the three individual years, 
but rather reflects a large degree of overlap in 
schools applying from year to year. This overlap is 
illustrated by noting that the 670 schools applying in 
1985 constituted 83 percent of all the schools that 
ever applied in 1985-87, the 508 in 1986 were 
63 percent of all applying, and the 494 in 1987 were 
61 percent of that same group. Of all eligible 
schools, 70 percent made no CSIP applications in 
any of the three years covered in this evaluation. 

Of the 811 schools that did apply, 410 (51%) 
received funding for one or more proposals at some 
time in the 1985-87 period. Within the period, 
school-level success rates increased gradually, from 
28 percent in 1985 to 47 percent in 1987. This year- 
to-year trend was also seen in the data for individual 
Pis and (again) reflects the increased CSIP funding 
that became available in 1987. 



Table 5-6. Rates of CSIP participation and success for 
eligible institutions. 1985-87 



index 


Grant year 


Total 


1985 


1986 


1987 


Percent of tne 2.706 










eligible schools that 










submitted any CSIP 










proposals 


30.0 


24 8 


18.8 


18.3 


Of schools that 










submitted proposals. 










percent with one or 










more funded 


50.6 


28.4 


34 8 


47.4 



Source Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment, NSF. 
1990 

It was noted earlier that individual Pis who persisted 
by submitting a revised proposal if their first effort 
failed often achieved success on the second try. The 
benefits of persistence in program participation were 
also evident at the institution level, where success 
levels were much lower for institutions that 
submitted only one CSIP proposal in 1985-87 (23%) 
than for institutions submitting more than one 
proposal (Figure 5-2). The latter group had a 
61 percent chance of receiving at least one award, 
and had a 30 percent chance of receiving two or 
more awards. 



Figure 5-2: Institutions' success in winning CSIP grants in 1985-87 as a function of number of 
proposals submitted 



Percent with 0 awards 



Percent with 1 award 



Percent with 2 awards 





Schools with only one CSIP proposal 



Schools with 2 or more CSIP proposals 
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Chapter 6 

CSIP Project Development and Implementation 



Highlights 

After a brief overview of the current status of 1985- 
87 projects at the time of the survey, this chapter 
describes the process of CSIP project development 
from notification of funding to full implementation. 

CSIP projects generally were implemented as 
proposed and in less time than the 30 months 
allowed by the program, reflecting well on the 
quality of the planning process at the proposal stage. 
Most 1985 through 1987 grant-year projects were 
close to full implementation at the time the 
questionnaires were completed, and the majority of 
project changes noted in the questionnaires appear 
to be expansions or other improvements. The few- 
Pis who decided on alternate equipment did so 
generally to better fulfill their curriculum 
development plans. 

The tasks involved in basic project setup were 
numerous and time consuming, requirng many 
hours of work by the PI and others, often including 
other faculty and students. Where the PI and other 
faculty dedicated themselves to the project, and the 
cooperation of the administration was obtained, 
implementation went most smoothly. 



Project Progress and Adherence to Schedule 

Most CSIP projects arc close to full implementation. 
The majority of projects in each of the grant years 
surveyed are mostly or fully operational, meaning 
that most equipment has been purchased and 
installed, and most course materials have been 
developed and tested with students (Table 6-1; 
Figure 6-1). Considering that 1987 projects were 
only a year and a half into their two-year (30 month) 
grants at the time of the mail survey, this is quite 
remarkable. In addition, only a handful of projects 
have made little or no progress. Underscoring the 
importance of planning for CSIP projects is a 
tendency for Pis who previously applied to fund the 
same project to be further along than those who 
were first-time applicants when funded. 



Table 6-1. Principal investigator's report o< the progress o< 
CSiP project by grant year: 1985-87 



Progress 


Grant year 


1985 


J 1986 


1987 


Total 


100.0% 


100.0% 


1 00.0% 


Fully operational 


88.0 


75.8 


37.0 


Mostly operational 


10.1 


12.1 


41.3 


Partly operational 


1.9 


11.0 


13.0 


Not yet operational 


0.0 


1.1 


8.7 



Source: Assessment of tne 1985-87 College Science instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990. 



Figure 6-1:-- Percent of CSIP projects that are fully or mostly operational: 1985-87 
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CSIP projects also closely follow the time schedules 
set out in proposals. Most Pis report that their 
projects arc on or ahead of schedule, and few report 
that their projects are behind schedule. In general, 
as projects have more time to progress they are 
more likely to be on or ahead of schedule: close to 
85 percent of 1985 and 1986 grantees report their 
projects are on or ahead of schedule, versus 
74 percent of 19S7 grantees (Table 6-2). Also, 
grantees who previously applied for CSIP funding 
for their projects are more likely to be on or ahead 
of schedule than those receiving grants on first 
application (85 percent versus 79%). This may be a 
function of the additional development time related 
to the preparation of two proposals. 

The Pi's institutional experience is also a factor in 
the timeliness of project implementation. If the PI 
has been a faculty member at the institution for only 
1 to 6 years, the project is more likely to be behind 
schedule (28%) than if the PI has been a faculty- 
member for 7 to 15 years (15%) or 16 years or more 
(16%). On the other hand, replacement of the PI 
seems to slow projects down: when the original PI is 
still overseeing the project, 82 percent are ahead of 
schedule or on schedule in contrast to 72 percent of 
projects where the original PI has been replaced. 

Departure from Original Proposal 

Equipment Changes 

Most projects closely follow the equipment plans 
proposed: there were virtually no significant 
changes reported between the equipment proposed 
for the project and that actually purchased. As seen 



in Table- 6-3, two-thirds purchased the exact 
equipment\ proposed, and another 32 percent 
purchased equipment that was functionally 
equivalent to that in their proposal. Less than 
2 percent of projects ur.ucrwent any substantial 
equipment substitution. 



Table 6-3. Principal investigator's report of equipment 
changes to CSIP project from original proposal 
by grant year: 1985-87 





Grant year 


Equipment purchased 


Total 


1985 


1986 


1987 


Total 


1 00.0% 


100.0%100.0°/c 100.0% 


Same as proposed 


66.6 


70 7 


72.5 


59.8 


Same function 


31 6 


26.0 


26.4 


39.1 


Equipment same/different 


4 


1.4 


0.0 


0.0 


Different equipment and 
purpose 


1.3 


1.9 


1.1 


1.1 



Source: Assessm&nl of the 1985-87 College Science Instrumentation 
Program. Oflice of Studies and Program Assessment. NSF. 
1990. 



The reasons for changes in the equipment purchased 
generally fell into four categories. One reason was 
to take advantage of an opportunity to buy more 
advanced or newer models of the proposed 
equipment. Some examples in the grantees* own 
words: 

■ Instrument purchased had superior spec's (optical 
performance, etc.) to originally proposed 
equipment, <£ manufacturer's discount made it 
affordable. 



Table 6-2. Principal investigator's report of the status of CSIP project by selected characteristics: 1985-87 



Project status 


Total 


Grant year 


Previous 
applicant 
status 


PI status 


Years on 
faculty 


1985 


1986 


1987 


Yes 


No 


New 


Ongina! 


1-6 


7-16 


16 or more 


Total 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 100.0% 


100.0% 


100.0% 


100.0% 


Ahead of schedule 


6.0 


8.7 


5.5 


4.3 


8.3 


5.2 


7.9 


5.8 


3.9 


7.9 


6.0 




74.8 


78.3 


79.1 


69.6 


76.9 


74 1 


64.6 


75.7 


68.5 


76.8 


78.3 




19.2 


13.0 


15.4 


26.1 


14.8 


20.7 


27 4 


18.5 


27.6 


15.4 


15.7 



Source: Assessment of the 1985-87 College Science Instrumentation Program. Office of Studies and Program Assessment. NSF, 1990. 
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■ Newer technology* made more sophisticated 
equipment available within our cost range, 

■ {Vendor} was willing to 'deal* for the IR & HPLC 
systems as a package so, coupled with a larger 
overmatch from the college, we were able to 
purchase gradient HPLC systems with an 
integrated rather the simple isomatic unit 
proposed. In addition, the ratio-recording model 
IR was supplied. 

■ / purchased a computerized physiobiological 
recording and biofeedback system instead of the 
conventional s\'stcm which I had planned to buy 
at the time my proposal was submitted. Hie 
computerized system employs more advanced 
technology, is more versatile, and is easier to 
operate. 

■ Altered equipment specifications in order to: I) 
save money, 2) achieve same original project 
objectives, 3) used savings to extend scope of 
project by purchase of additional equipment. 

■ Between the time we submitted and were 
consequently awarded the CSIP funding DEC 
came out with the Microvax II To replace the 
Microvax I. T)\c latter was in our original 
proposal, but we ended up purchasing the newer 
(and better) Microvax IL 

A second reason was the location of a more 
economical source for the proposed equipment, 
resulting in the ability to purchase a more advanced 
instrument or additional components: 

■ Was able to get a large enough discount for a more 
expensive instrument. 

■ Different vendor had better equipment-improved 
design. 

■ Different equipment due to state's low bid 
procedure. 

m Lower prices allowed us to outfit two computers 
instead of one. 

■ Cost bidding for equipment allowed us to 
purchase additional instrumentation. 

■ A change in vendor allowed us to purchase 12 
phase contrast microscopes at a lower cost- 
allowing the purchase of a refrigerated table-top 



centrifuge. We substituted a dcionizing system 
instead of requested on the advice of the 
manufacturer. 

A third reason was thai, as some projects were 
implemented, the Pis learned more about the 
equipment and found that modifications or 
expansions made the project more workable or 
would allow more students to participate: 

■ We changed the type and number of computers 
purchased, so that each lab group can have a 
computer TJiis makes a system that is much 
better than what was proposed. 

■ We had a compatibility problem between the 
Tektronix terminals and the proposed monitors for 
the classroom. We substituted an overhead 
projection system which cost about $10,000 more 
than proposed. 

■ Hie scanning electron microscope required some 
unanticipated modifications to facilitate use of the 
EDS system. 

■ Because the project included about 100 separate 
components, it required extensive discussion and 
testing to put together the best set of components. 
Some changes were required. 

■ Hie equipment proposed initially proved to be 
cumbersome in column changing and better suited 
to functioning as a dedicated process instrument. 
We are very happy with the alternate instruments 
we chose. 

■ By ordering a more economical printer/plotter, we 
were able to purchase a demo auxiliary computer 
system thus enabling two students to use the 
system at the same time-one to acquire IR spectra 
while another is using the auxiliary computer for 
data interpretation. 

And, in some cases, equipment changes were 
necessitated by circumstances beyond the control of 
the PI: 

■ Some equipment that we hoped to purchase 
became unavailable. We used the money to 
purchase (better) components and we are having 
one recently hired machinist assemble/build these 
devices. 



Vie proposed equipment (some of it) is no longer 
manufactured. Vie replacements were more 
expensive, requiring some changes in the plan. 

Change in faculty caused consequent alteration in 
focus of advanced labs and student research. 

Failure of my university to provide adequate lab 
space (as promised) has necessitated not 
purchasing one piece of equipment. 



Other Project Changes 

Other changes between the proposed and imple- 
mented projects were also minimal (Table 6-4). 
Only about IS percent of projects made changes to 
their originally proposed curriculum development 
plans, and 20 percent made changes to their 
originally proposed course content. About 1 in 10 
projects made changes to their original plans in the 
areas of format, audience, and course level. The 
proportions making plan changes tends to be slightly 
lower for more recent years, perhaps indicating that 
changes evolve over time, as the PI has more 
experience running the project or the 
implementation becomes more complete. 



Table 6-4. Principal investigator's report of other changes 
to CSIP project from original proposal by grant 
year: 1985-87 



Changes 


Grant year 












Total 


1985 


1986 


1987 



Total 100 0% 100.0% 100.0% 100.0% 

Curriculum 18.1 25.5 14.3 15.2 

Format 11.2 15.4 12.1 7.6 

Content 20 1 23.6 25.3 14.1 

Audience 10 0 12.5 11.0 7.6 

Course level 9.0 10.6 7.7 8 7 

Note: Percents may not add to 100 because not all Pi's reported 
changes 

Source Assessment ot the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF, 
1990. 

Generally, respondent explanations of the changes 
made in their CSIP project plans fell into three 
categories. The most frequently mentioned had to 



do with expansion of the project in terms of content 
or the number or level of students reached: 

■ Wiiat we could do with the equipment became 
more clear as wc actually used it. Vie lab 
exercises, in format and content (and thus the lab 
curriculum), evolved with the project. 

■ Instruments purchased with these funds are being 
used in more courses than originally proposed. 
Viey have had a greater impact than originally 
proposed, Wc now have sophomores in 
quantitative analysis using the GC*s. 

■ Wc arc integrating the equipment throughout the 
curriculum from freshman to senior level courses. 
Originally, the project was proposed for upper level 
courses. 

■ Student laboratory involvement with the FT/IR 
has been more intense than originally planned in 

*y the physical chemistry and advanced inorganic 
courses. More laboratory time is devoted to IR 
studies of molecules. Additionally, a heavier 
introductory emphasis is planned for the course of 
first FT/IR contact, introductory organic 
chemistry. 

■ Due to student feedback the proportion of time 
spent in lab sessions has been increased beyond 
what the original proposal outlined. Students have 
across the board requested more time for lab 
sessions or extended labs. Vie extent to which 
students in experimental psychology have utilized 
the equipment is much greater than I would have 
estimated. 

■ More of the advanced mathematics courses are 
using the equipment, especially the geometry 
courses. More chemistry professors have used the 
equipment and have since included more graphic 
uses in their classes. 

■ Considerably more use of computer interfacing 
with chromatography in analytical courses than 
anticipated. 

However, about a third of the Pis who reported 
expansions wrote that implementation required 
changes and compromises which narrowed the scope 
of the project or changed its content: 

■ The project was written by a physical chemist who 
had intended to incorporate use of NMR into that 
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course. He has since left the university so those 
aspects did not get developed. 

■ Content changes almost every semester because 
different faculty teach the courses each year and 
each prefers to assigti their own labs. 

■ Hie expected use of the instrument by faculty for 
classroom instruction has not entirely developed; 
faculty bring their classes for exercises but expect 
the technician to actually do the instrument work 
for the class. 

m Have not integrated equipment into some courses 
in which wc proposed to use the instruments. 

Finally, a number of grantees indicated that, in 
addition to its instructional use, the equipment was 
also being used for faculty and student research 
activities: 



Table 6-5. Principal investigator's report of time needed to 
implement CSIP project by grant year. 1985-87 





Grant Year 


Task 


Total 


1985 


1986 


1987 




Hours 


Hours 


Hours 


Hours 




336 6 


373 7 


322.7 


3G4 7 


Order, install, learn to 
use equipment . . 


129 9 


126.0 


139 1 


126 5 


Develop course 
materials 


140 6 


127 8 


132 4 


154.5 


Train staff 


44 1 


42.4 


38 5 


48 7 


Integrate equipment 


76.6 


73.0 


64.4 


87.7 



Source - Assessmenl of the 1985-87 College Science instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990. 



■ More use of the equipment by faculty and students 
working on research projects. 

■ Tlic greatest use of the instrument has not been in 
class instruction but in undergraduate and 
graduate student research. 

■ Hie equipment, ultracentrifugc and liquid 
scintillation counter, arc proving difficult to 
introduce into our student labs. TJiercforc we are 
using them, at present, mostly for student research 
projects. Wc arc continuing to explore methods for 
introducing them into student labs. 

Setup Work Needed and Proportion Completed 

CSIP projects require a major commitment of time 
and effort on the part of the PI. The average time 
needed to order, install, and learn to use the 
equipment, develop course materials, train staff, and 
integrate the equipment into the curriculum was 337 
hours, or 42 8-hour days (Table 6-5). About 
40 percent of all respondents reported that the 
amount of time needed to implement their projects 
was substantially greater than they expected when 
they submitted their proposals (Table 6-6). 



Table 6-6. Principal investigator's report of time provisions 
for CSIP project implementation by grant year. 
1985-87 



Grant year 



Time factor 


Total 


1985 


1986 


1987 


Total 


100.0% 


100.0% 


100.0% 100 0% 


Time more than anticipated 


39.5 


35.6 


39.6 


42.2 


Requested release time 


23.3 


19.2 


25.3 


25.0 


Release time granted 


19.2 


12.5 


20.9 


22.8 


Other provisions made 


20.1 


18.8 


20.9 


20.7 



Note: Percents do not add to 100 because respondents could 
indicate more than one or no time factors 



Source: Assessment of the 1985*87 College Science instrumentation 
Program. Office of Studies and Program Assessment. NSF, 
1990. 

The two most time consuming tasks were ordering, 
installing, and learning to use the equipment, and 
developing course materials, each of which took an 
average of 17 days. Integrating the equipment into 
the curriculum was next most time consuming, with 
about 70 percent of projects reporting an average of 
10 days spent on activity in this area. Half of 
projects reported staff training activities, which 
averaged six days to complete. 

Less than a quarter of principal investigators 
requested release time to implement their projects. 
Pis appear to have a general idea of the effectiveness 
of such requests at their own institutions since 
82 percent of those asking for release time received 



it. Many other Pis were of the opinion that such 
requests were a waste of time. Other provisions, 
which could include summer salary or the hiring of 
technicians, were made for about 20 percent of 
projects. 

Although Pis are solely responsible for oversight of 
project implementation, the substantial involvement 
of other people also contributed to the timely 
implementation of projects. Pis reported an average 
of nine people involved in project setup (Table 6-7). 
Almost two-thirds (64%) of projects involved 
participation by other faculty members; more than 
half (56%) involved students; 43 percent involved 
technicians; one-third involved lab assistants; and 
about a quarter (27%), other university employees. 
Also involved (though infrequently) were: outside 
contractors, company representatives, vendors, and 
consultants. 



Table 6-7 Principal investigator's report of number of 
people involved in CSIP project setup by grant 
year: 1985-87 





Grant year 


Personnel 




1985 


1986 


1987 


Total 


Total mean numDer 


8.8 


9.6 


8.2 


8.7 


PI/CO-PI 


1.4 


1.3 


1.3 


1.5 


Other faculty 


2.4 


25 


2.5 


24 


Students 


6.3 


8.8 


5.1 


5.7 


LaD assistants 


2.3 


2.8 


1.8 


2.1 


Technicians 


1 4 


1.2 


1.6 


1.5 


Other university employees . 


2.3 


2.4 


2.4 


2.2 


Outside contractors 


1.8 


1.7 


1.7 


2.0 



Source Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF, 
1990. 



The setup work for most projects has been 
completed. About 87 percent of the work in 
ordering, installing, and learning to use the 
equipment has been done, as has 75 percent of the 
work in developing course materials, 79 percent of 
the work in training staff, and 74 percent of the work 
in integrating the equipment into the curriculum 
(Table 6-8). The proportion of work completed 
drops off substantially for 1987 grant year projects, 
where, for instance, only 65 percent of the work in 
developing course materials has been completed. 



Table 6-8 Principal investigator's report of portion of CSIP 
project work completed at time of survey by 
grant year 1985-87 



Task 


Grant Year 


Total 


1985 


1986 


1987 


Order, install, learn to 










use equipment 


86.8% 


92.8% 


89.8% 


80.5% 


Develop course 










materials 


74.9 


86 3 


77.8 


64.9 


Tram staff 


78.9 


86 4 


85.5 


69.9 


Integrate equipment ... 


74 4 


87.1 


74 1 


63.6 



Source Assessment of the 1985-87 College Science Instruments 
tion Program. Office of Studies and Program Assessment. 
NSF. 1990 



Grantee comments on factors affecting the time for 
project implementation generally fall into three 
categories. The first pertains to the acquisition and 
integration of the equipment itself: 

■ Simply getting curriculum changes through our 
coordinating board (was time consuming). 

m Initial efforts in achieving operational status of the 
eqiupment was greater than expected. I learned 
both how to operate the instrument and maintain 
and repair it. Many experiments were developed in 
order to find the best ones for students and to 
explore the capabilities of the instrument 
(GC/MS). 

■ Hie multitude of software available took many 
hours to evaluate; I needed to evaluate the 
software in order to determine which type of 
computers to purchase for students (not all 
software is available for all computers, obviously). 
Wien I saw how much time this would take, I 
asked for release time (which I did not get). 

■ Installation was a major headache! It literally 
took months. 

m Spent great amount of time evaluating equipment, 
seeking vendor support for matching funds. None 
were forthcoming. Learning to use the equipment 
took much longer than expected. University 
computing service supported project from start and 
are continuing to do so. 

■ More time in training faculty on use of HPLC 
than foreseen. 
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■ Tlie equipment is not reliable; when we had it set 
up, we had to call in the vendor several times for 
adjustments. It is also not easy* to learn to use. 
Since there has been no release time for this, it has 
been disconnected until Summer of 1989 when 1 
expect to have time to work with it. 

■ / simply didn't realize how much time it would 
take to research, order and help install, debug and 
implement my equipment. 

■ Learning to use the equipment was easy; 
integrating it into the curriculum v.'as more difficult 
( we were used to the old ways!). 

■ Tlie project became more complicated than 
originally proposed. Equipment purchased was 
more complex <£ greater variety of software had to 
be learned. Tixrce faculty members applied for and 
received one month's salary each for curriculum 
development related to this project. 

Staff commitment and availability was a second 
factor affecting the timeliness of project 
implementation. It was frequently noted in 
respondent comments: 

■ No release time granted by the dean. 

■ University granted release time to organize and 
write laboratory manual. Tlie university provided 
money for maintenance of equipment. 

■ Tliree students were hired to help write some of the 
computer programs. 

■ PI was given a single course load reduction one 
semester to set up the lab. No help was given with 
the physical setup of lab. All tables, security 1 , 
equipment, etc. were installed by the PI. 

■ We have received a few weeks of summer support 
for work on computer projects: building hardware 
interfaces, writing software. Tlie support ha\ been 
minimal and is not nearly enough. 90% Of the 
work has been on our own time. 

■ / used part of my sabbatical leave to implement 
the project. Also used summertime (unpaid). 

■ The PI utilized substantial amounts of his summer 
(and that of a student) time to do much of the 
initial setup work on this project 



■ / obtained 3 course releases (one over each of 
three quarters) to work on the integration of the 
equipment into the curriculum. Tlie hardware wc 
purchased was more complicated than expected 
and the documentation was inadequate for our 
purposes, so the time spent was greater than 
anticipated. 

■ / requested and was granted a one course 
reduction for the 1987-88 school year. However, 
an unexpected change in staff cancelled those 
provisions. 

■ Because the ordering of equipment was complex, 
and because some new skills were required of mc t 
I was unable to implement the project until I 
requested and was granted a sabbatical leave for a 
semester. 

■ // has been very difficult to generate sufficient time, 
of a continuous nature, to make real progress. 
Tliis problem related to departmental staffing and 
other major university responsibilities. Proposal 
for special scholarly leave time for CSIP project 
was not granted. 

■ One faculty 1 member took one of the computers 
home and spent much extra time at home 
developing the necessary software. 

Finally, a few respondents cited administrative 
factors as affecting the implementation of their 
projects: 

■ University 1 regulations required that all equipment 
costing more than $100.00 be let out for bid. Tliis 
was a very time consuming un necessity and 
significantly delayed project start up. 

■ It has taken longer than expected to integrate a 
solid state physics lecture course with the solid 
state physics experiments. Tliis delay was caused 
by the inability of the Mechanical Engineering and 
Engineering Science and Physics departments to 
agree who should teach the solid state physics 
lecture course. Tliis issue has been decided in 
favor of the physics department by the 
administration. 

■ Extensive renovations of our science building were 
complete a year after we obtained our CSIP 
equipment. As a result, we had to install the 
equipment twice: once in a temporary facility and 
later in a permanent location. 



Kinds of Problems Encountered 

Fewer than half of the Pis encountered major 
problems with project implementation. Overall, 
44 percent mentioned major problems with their 
projects (Tabic 6-9). Problem areas most often 
mentioned were ordering, installing, and learning to 
use the equipment (21%), and maintaining the 
equipment (12%). In fewer than 10 percent of 
projects, developing course materials, training staff, 
integrating the equipment into the curriculum, 
operating the equipment, and other aspects of 
project implementation were mentioned as 
problems. 



Table 6-9. Principal investigator s report of major problems 
encountered in CSIP project implementation by 
grant year- 1985-87 





Grant year 


Problem area 












Total 


1985 


1986 


1987 



Total 

Any problem 

Ordering, installing, 
learning !o use 
equipment 

Developing course 
materials 

Training staff 

Integrating equipment 
into curriculum 

Maintaining 
equipment 

Operating 
equipment 

Other aspects 



100.0% 100.0% 100.0% 100 0% 



44 4 37.5 



41.8 



51 1 



20 6 


16.8 


16.5 


26.1 


80 


6.7 


3.3 


12.0 


4.2 


3 4 


3.3 


54 


7 8 


8.2 


7 7 


7.6 


12.3 


14.9 


13.2 


9.8 


6.8 


4.6 


7.7 


7.6 


9.2 


5.3 


11.0 


10.9 



Note: Percents do not add to 100 because respondents could 
indicated more than one or no major problems 

Source Assessment of the 1985-87 College Science Instrumentation 
Program. Office of Studies and Program Assessment. NSF. 
1990. 
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Chapter 7 



assessment of the application Process 



Highlights 

Most aspects of the CSIP program administration 
were judged favorably by both grantees and 
unsuccessful applicants, but the timing of 
announcements and the deadline for submission, 
and (particularly for applicants) the clarity of review 
feedback were identified as areas needing 
improvement. The primary area of concern noted 
by applicants was the clarity and appropriateness of 
the program guidelines. Additional 
recommendations concern-ing the value of 
provision for training support in grants and for 
provision for instrument price increases were made 
by Pis during site visits. 



Overview 

Respondents to both the grantee and unsuccessful 
applicant mail que tionnaires were asked to rate 
eight aspects of CSIP program administration as 
adequate or needing improvement: 

« Timing of announcements; 

« Deadline for proposal submission; 

■ Dissemination of program announcements; 



■ Clarity of CSIP program guidelines; 

■ Appropriateness of CSIP program guidelines; 

■ Budget detail requirements for the proposal; 

« Narrative detail requirements for the proposal; 
and 

■ Clarity of feedback from proposal review. 

Supplementing the questionnaire data are data 
from responses to a question asked during site visits 
about ways the PI thought the CSIP program could 
be improved. 

The most striking finding of the analysis of the 
questionnaire data is that grantees and unsuccessful 
applicants have few criticisms of the CSIP 
administration process. Grantees, on average, 
assigned less than one negative rating for the eight 
elements, and no single aspect of program 
administration received a negative rating from as 
many as one-fifth of grantees. Unsuccessful 
applicants assigned less than two negative ratings 
each, but five aspects received negative ratings from 
more than one-fifth of that group. Clarity of review- 
feedback was by far the most criticized aspect, with 
over 50 percent of unsuccessful applicants rating 
this as in need of improvement (Figure 7-1). 
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Figure 7-1: Elements of grant process and program needing improvement 



Timing of announcements 

Deadline for proposal submission 
Dissemination of program announcements 
Clarity of CSEP program guidelines 
Appropriations of CSEP program guidelines 
Budget detail requirements for the proposal 
Narrative detail requirements for the proposal 
Clarity of feedback from proposal reveiwers 
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Those respondents who rated aspects as being in 
need of improvement were asked to provide 
explanations or recommendations based on their 
experience with the CSIP program. Although most 
grantees and unsuccessful applicants did not see a 
great need for improvement, the inputs of those 
who saw such a need provide important 
information. These explanatory comments give a 
clear view of the perceptions of CSIP's weak points 
by those who submit proposals, and the ways they 
would like to see the program change.. In the 
remainder of this chapter responses to each aspect 
will be discussed, and statements from grantees and 
unsuccessful applicants will be presented. 

Timing of CSIP Program Announcements and 
Deadline for Proposal Submission 

Approximately one in seven grantees ai d one in 
four unsuccessful applicants saw a need for 
improvement in the timing of announcements. 
There were slightly more negative responses to the 
deadline for proposal submission area, but 
responses follow a similar pattern. Negative 
responses were given by 18 percent of grantees and 
30 percent of unsuccessful applicants. 

Written comments from grantees and unsuccessful 
applicants note very similar problems: respondents 
reported that the time between the announcement 
and the submission date is inadequate, and the 
submission deadline fell at a bad part of the 
academic year. 

From Grantees 

■ Tlircc months should be allowed between these 
dates {announcements and deadlines} for 
continuing programs; more time if there is a 
substantial change in announcement from 
previous years. 

■ Tlie old January* deadline was a problem. Tlie 
new deadline is even tougher. Couldn't it be 
moved to spring? 

■ Not enough time is allowed between program 
announcement and proposal submission 
deadline. (I suspect Ms may be a problem for 
those unfamiliar with the program) 



■ Tlie early November deadline with early fall 
announcement is a time-problem for those who 
are not aware that one must anticipate this 
deadline at least 6 months in advance. 



From Unsuccessful Applicants 

■ Department chairpersons usually receive the 
announcements 6-8 weeks prior to the deadline. 
Tliis does not allow sufficient time for a faculty 
member or committee, with many other duties, to 
analyze needs in accordance with the guidelines, 
gather data, write the proposal, secure the 
appropriate university approvals and signantrcs 
prior to the deadline, 

■ Because of academic schedules, it would be 
useful to have the summer to prepare proposals. 
A separate, standard form could be used to gather 
information about the institution and department. 
Allow the narrative to focus only on the specific 
instrument proposal. 

■ In the case of small institutions, the faculty 
member alone must prepare the proposal. We do 
not generally have a number of prepared 
proposals to draw from. We need time. 

■ By the time I get an announcement in the mail, 
there is, at most, J month before the deadline. 
Tliis is not enough time. 

Dissemination of CSIP Program Announcements 

Only 7 percent of grantees and 14 percent of 
unsuccessful applicants found problems with the 
dissemination of announcements. Again, the focus 
of comments for the two groups was similar, 
centering on the difficulty in receiving 
announcements, and the suggestion to mail them 
directly to departments rather than to development 
offices was made frequently. 



From Grantees 

■ Finding out about grant funding is difficult. 
Seems to be a problem with mailing lists. 
Perhaps would be better to send to faculty- 
members or department heads than to 
development officers. 

■ Small schools have less contact with NSF and 
tend to not hear about new programs as quickly. 
Information did not come to me. Our institution 
docs not receive all announcements. J had to 
seek the information. 

■ Prior applicants should get new guidelines 
automatically. 

■ Materials are not directly forwarded to 
department heads, but go to the college 
administration. Send directly to departments as 
well as research office. 

From Unsuccessful Applicants 

■ Some years I have received announcements, but 
in other years J have not. 

■ / called several time before receiving an 
application packet. 

■ Mail announcements directly to previous 
applicants, especially first-timers. 

■ We have had to call NSF to get a copy of the 
guidelines. Guidelines should be out by August 
to allow adequate time to prepare a proposal. 

Clarity and Appropriateness of CSIP Program 
Guidelines 

Only 4 percent of grantees felt that CSIP guidelines 
were not sufficiently clear, and only 5 percent felt 
they were not appropriate. Many more 
unsuccessful applicants felt this way: 26 percent felt 
they were not c^ar, and 22 percent felt they were 
not appropriate. This finding is consistent with the 
analysis in Chapter 5 of this report, which found 
that proposals that more consistently promote CSIP 
goals have a greater chance of funding. 
Understanding and following the program 



guidelines is probably very important to writing a 
successful proposal. 

Recommendations from grantees and unsuccessful 
applicants focus mainly on specifying/changing the 
kinds of equipment/projects that the CSIP program 
will fund, and the types of institutions that should 
be given funding priority. 

From Grantees 

■ Guidelines push department towards equipment 
it probably docs not need. Distorts purchases for 
other areas of the department. 

■ It is not uncommon to find conflicting statements 
within the guidelines. 

■ Guidelines arc OK for someone who has some 
experience but arc not of great assistance to the 
first time grant writer, especially at small 
institutions where there is no grants office. 

■ Guidelines do not correspond very well to 
guidelines provided by NSF officials informally 
by phone. We would probably not have been 
successful if we had relied on only the vague 
written guidelines. 

■ A proposal shouldn't have to be based on a 
sensational new idea to be funded. Basic 
equipment needs are obvious, and a thorough 
proposal explaining how instrumentation will be 
used, even if nothing "new and exciting' 1 is being 
done is still worthy. 

■ / have on several occasions applied for CSIP 
grants. When I have not been successful a 
common response is that the proposed project is 
too sophisticated for undergraduates. I wish 
some policy could be developed so that decisions 
made by reviewers would not be so conservative 
about new innovative approaches. 

From Unsuccessful Applicants 

■ Guidelines need to define upgrading ~ docs it 
mean model of equipment or upgrading of lab 
content of exercise? 



■ Guidelines did not make it clear (he kind of 
equipment (student microscopes) I was 
requesting would be more appropriately funded by 
my university. 

■ Proposal writers need a clearer picture of the 
criteria used by the proposal reviewers. 

■ Guidelines need to be more specific as to what is 
eligible, e.g., if routine, standard equipment isn f t 
eligible ~ then the guidelines should state that 
only expensive or exotic equipment is eligible. 

m I feel that there is often a hidden agenda in the 
evaluation of proposals and that the real criteria 
for evaluation should be published in the 
guidelines. Departments with no experience of 
success can spend enormous effort with little 
chance of success unless they got input from 
persons with a history of success. 

■ Ttie guidelines do not stale that numerical 
student impact will be a major factor in awarding 
grants or that reviewers should consider it. TJiat 
was the only negative comment received on my 
proposal and I saw it was not funded. 
Guidelines must be improved. 

m I sense that the focus is to make good programs 
excellent - we need help to get from mediocre to 
adequate. 

■ As long as the eqidpment is primarily used for 
undergraduates a large university can apply for 
the program. That's simply not fair to small, 
undergraduate, teaching schools. We cannot 
effectively compete with the large schools! 

■ { CSIP) seems to indicate a bias toward 
computer related projects at the expense of 
standard or routine engineering equipment in 
short supply. 

■ Specify that "creative'' uses of equipment arc 
requested - no! just instrumentation to strengthen 
one's curriculum. 



Budget and Narrative Detail Requirements for the 
Proposal 

Few grantees (2% and 4%, respectively) felt thai 
budget or narrative detail requirements for the 
proposal were a problem. More unsuccessful 
applicants (109c and 13%, respectively) thought 
these areas needed improvement, but these aspects 
were also the ones that concerned unsuccessful 
applicants least. Few respondents wrote on this 
topic. A rare comment comes from a grantee: 

a Very detailed budget required, but with inflation 
PI loses much buying power. 

Clarity of Feedback from Proposal Reviewers 

Opinions about the clarity of feedback iron) 
proposal reviewers have the widest disparity 
between grantees and unsuccessful applicants (11% 
negative versus 51% negative, respectively). This 
aspect has the third highest negative rating by 
grantees, but has by far the highest negative rating 
by unsuccessful applicants (more than 20% higher 
than the next highest aspect). The concentration of 
unsuccessful applicants on criticism in this category 
merits careful consideration, especially since their 
narrative statements do not seem to be of the "sour 
grapes' 5 variety. Most were thoughtfully expressed 
and lacking in bitterness. Both grantees and 
unsuccessful applicants focused on the types of 
feedback they would like to receive, particularly 
with regard to consistency of comments by 
reviewers, and how reviewers should be selected. 



From Grantees 

■ Most unsuccessful proposals are resubmitted; 
comments on how proposals could be 
strengthened should be emphasized. 

m The reviewers should be selected with more care, 
and proposals should be reviewed by reviewers 
from similar institutions. Hot shots in research 
institutions will not usually see the validity of 
need for certain equipment items needed by a 
small teaching school when they themselves take 
such items for granted. 
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The consistency* of reviewers is a problem. The 
aspects of my proposal which the first reviewers 
criticized, were praised the next year when my 
proposal was funded. 

First submitted - not funded - excellent review. 
Same grant - resubmitted following year — terrible 
reviews and funded (no consistency in process). 

First time we were turned down and found 
reviewers comments very 1 general. Hie more 
detailed the reviewers comments, the more 
helpful it is to prepare for resubmission. 

From Unsuccessful Applicants 

Reviewers need to be consistent; it was obvious to 
us one reviewer did not appear to have read the 
proposal, other reviewers comments contradicted 
one another. One would praise the grant for an 
attempt at a specific objective while another 
complained that the same objective was not tried. 

Hie comments in the first review suggested 
specific areas where changes might make the 
proposal more acceptable. However, the second 
set of reviewers (the second year) criticized the 
proposal primarily for the things that were 
changed to respond to the first review comments. 
The guidelines and review process should be 
objective enough that this yo-yo-ing would not 
occur. 

There is the usual problem th : one cannot 
respond to criticism the year it is offered. The 
next year completely new reviewers may be 
reading the grant. Can one get a list of what 
equipment was funded for what project as a 
guideline? 

For CSIF proposals in engineering it is essential 
to include practicing engineers from private sector 
to ensure that the proposed instrumentation will 
promote the current professional practice. 

From my point of view, it seemed as though the 
reviewers were more interested in the existing 
equipment at the institution and the background 
of the applicant than the quality of the proposal 



■ It appears that most of the scientists reviewing the 
proposals have never worked in any segment 
except the educational segment. People with such 
a limited background or experience have limited 
knowledge in understanding what graduates need 
to be adequately prepared in entering the 
industrial world. Tliey fail to view proposals and 
evaluate them properly if the proposals do not 
address the request of some large and expensive 
piece of equipment that is usually designed for 
research rather than for teaching students the 
scientific skills of instrumentation and technology' 
needed for adequate professional perfonnance in 
the industrial and governmental segments of 
society. 

■ Reviewers did not react to my plan with respect to 
the goals, but rather their preconceived ideas of 
what should be done, 

■ It is imperative that an archaeological proposal 
not be sent to anybody in social sciences, since 
disciplines under the category of social science 
are totally unfamiliar with archaeology. Only 
archaeologists should review archaeological 
proposals, not philosophers, psychologists or 
language teachers. 

m Would like a variety' of reviewers. (J) from 
different regions, (2) from small colleges. 

Viewpoints Expressed During Site Visits 

During site visits Pis of operating projects were 
asked for suggestions about how the CSIP program 
might be improved. Several of their 
recommendations are not seen in data from the 
mail questionnaires, including allowing provision 
for training support and for price increases in 
instruments: 

■ PI feels that NSF should consider small training 
grants to enable PPs and other staff to learn to 
use the equipment (perhaps the year after it is 
acquired). 

m Suggests that NSF consider allowing training 
costs on sophisticated equipment as an allowable 
one-time part of the budget. {The PI and co-PI] 
would both like to see an instrument specific 
(NMR in this case) newsletter with lab manuals, 
etc. f to reduce duplication work. 



Project got caught in major marketing/price 
change that resulted in $15,000 higher cost to 
college than anticipated. Tliis might have been 
avoided by faster turnaround between 
application, award. 

The PI raised the issue of the handling of price 
increases in equipment which may occur over the 
review period. WJiilc recognizing that the review 
process requires sufficient time, he suggested that 
some provision for legitimate price increase be 
included in the budget. 
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CSIP SITE VISIT CONSULTANTS 



Dr. Charles Alexander 

Department of Electrical Engineering 

Temple University 

Dr. Richard Blatchly 
Chemistry Department 
Williams College 

Dr. Joanne Cameron 

Department of Cell and Structural Biology 
University of Illinois - Urbana 

Dr. Joseph T. Chowattukunnel 
Department of Biology 
Indiana University - South Bend 

Dean James Diefenderfer 
California State University - Fullerton 

Dr. Norman Henderson 
Severance Laboratory 
Obcrlin College 

Dr. Timothy Keiderling 
Department of Chemistry 
University of Illinois - Chicago 

Dr. J. H. Kreiner, Chair 

Department of Mechanical Engineering 

California State University - Fullerton 

Dr. Karen W. Morse, Dean 
College of Science 
Utah State University 

Dr. Steven Murray 

Department of Biology 

California §t£te University - Fullerton 



Dr. Edwin Robinson 

Department of Geological Sciences 

Virginia Polytechnic Institute 

Dr. Paul Schatz 
Department of Chemistry 
University of Wisconsin - Madison 

Dr. Herbert Silbcr 
Department of Chemistry 
San Jose State University 

Dr. Donald Sloan 
Department of Chemistry 
CCNY 



Dr. Bruce Thomas 

Department of Physics and Astronom 
Carleton College 

Dr. Reed Warren 
Department of Biology 
Utah State University 

Dr. Janice Woodall 

Southern Union State Junior College 

Dr. Sandra Yorka 
Department of Physics 
Denison University 
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PRETEST SITE VISIT SCIENTISITS/CONSULTANTS 



Dr, Charles Detwiler 
Department of Biology 
Houghton College 

Dr. Jean Rogers 

Department of Computer Science 
Stanford University 

Dr. Dennis Steele 

Department of Computer Science 

University of Wyoming 

Dr. Robert Ziller 
Department of Psychology 
University of Florida 
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ASSESSMENT OF THE COLLEGE SCIENCE INSTRUMENTATION PROGRAM 

GRANTEE QUESTIONNAIRE 

National Science Foundation 



The National Science Foundation is conducting an assessment of the College Science Instrumentation 
Program (CSIP). The purpose of this survey is to determine the impact of CSIP on the participating 
institutions, their students, and on efforts to improve undergraduate science instruction. 

We ask that the requested information be obtained from the current principal investigator, or, if this is not 
possible, from the person who is most knowledgeable about the history and current status of the project. 
Your name and address and the CSIP project title and year of project from the original proposal appear 
below. Please correct the label if any of the information is incorrect. 



Where exact cost (or other) data are not available, estimates are acceptable. (Your estimates will be better 
than ours.) 

Ail information you provide is confidential and will be published only in aggregate form. Your response, 
though important for an accurate assessment, is voluntary, and failure to provide some or all of the 
information will in no way affect you or your institution. This information is solicited under the authority of 
the National Science Foundation Act of 1950, as amended. 

Pleas* return this form by Dacambtr 22. Your cooperation in returning the survey questionnaire 
promptly is essential to the timely completion of the assessment. Please return the completed survey to: 

WESTAT, INC. 
1650 Research Blvd 
Rockville, MD 20850 



If you have any questions regarding this survey, please contact Ms. Carin Celebuski at Westat's toll-free 
number 800/937-8281, or contact Mr. David Florio of NSF at 202/357-7425. 



Public rtporting burdtn for tfits coJlaction of information is astimatad to avaraga 35 minutas par rasponsa, including tht tt.nt for 
raviawing instructions, Marching axisting data sources, gathering and maintaining tht data needed, and competing and reviewing 
the collection of information. Send comments regarding this burden estimate of any other aspect of thia collection of information, 
including suggestions for reducing this burdan, to: 



Herman G. Reming 
Clearance Officer - Room 208 
Division of Personnel and Management 
National Science Foundation 
Washington, O.C. 20550 



and to 



Offioa of Management and Budget 
Paperwork and Reduction Projact (3145-01 13) 
Washington, D.C.. 20GO3 
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I. PROGRESS 



This first section, questions 1 through 9, will gather information about the progress, problems, and 
current status of your CSIP project. NSF will use this information to see how its programs might be made more 
responsive to the needs of institutions like yours. Later sections of this questionnaire will deal with project 
finances, impacts, and demographics. 

1. Please check the item below that best describes your project in terms of equipment, course materials 
and student exposure. 

□ Not yet operational: Equipment not yet purchased 

OR 

Equipment not yet Installed 
OR 

Course materials not yet developed 

□ Partly operational: Most equipment purchased and installed 

BUT 

Course materials not yet fully developed 
OR 

Untested with students 



□ Mostly operational: 



Most equipment purchased and installed 
AND 

Most course materials developed 
BUT 

Not yet fuily tested with students 



2. 



□ Fully operational: 



Most or ail equipment purchased and installed 
AND 

Most or ail course materials developed 
AND 

Fully in use with students 



if your CSIP project Is not yet fully operational, please estimate how long it will take to become fully 
operational. 

MONTHS 
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3. Please assess your progress to date in implementing your CSIP project. 

□ Ahead of schedule 

□ On schedule 

□ Behind schedule 

Questions 4 and 5 will aid in assessing the amount of change projects have undergone between proposal 
and implementation, and the reasons for changes. 

4. Since your proposal, have there been changes in the equipment ordered or planned for your CSlP 
project? Check the item below that best describes your equipment situation. (A change in vendor does 
not constitute a substantive change.) 

□ Entirely or mostly as proposed 

□ Different equipment but functionally equivalent to that proposed 

□ Same equipment but serving a different purpose on the project 

□ Equipment and Its function substantially different than proposed 

Please explain any changes: 



Aside from changes in equipment, have the following aspects of your project evolved or been modified ii 
substantia] ways since the award was made? 







YES 




a. 




□ 


□ 


b. 




□ 


□ 


c. 




□ 


□ 


d. 




□ 


□ 


e. 




□ 


□ 


f. 


Oth«r (SPECIFY! 


□ 


□ 


C- 


Please elaborate on *YES" answers to 5a through 5f. 
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6. Please Indicate how much time has been spent (or will be spent) on the following setup tasks for your 
CSIP project, and the current status of each. 

APPROXIMATE PERCENT 
NOT HOURS OF TASK 

APPLICABLE FOR TASK COMPLETE 



a. Ordering, installing and 

learning to use the equipment % 

b. Developing course materials 
(e.g., outlines, syllabi, lab 

manuals, lab experiments) □ % 

c. Training staff (e.g., 
technicians, lab monitors, 

lab assistants) O % 

d. Integrating the equipment 

into the curriculum □ % 



7. How many of each of the following categories of people were (or will be) involved in the setup work for 
the project? (IF NONE, WRITE IN '0'). 

NUMBER 

a. P.l./Co-P.l.'s 

b. Other faculty 

c. Students 

d. Lab assistants 

e. Technicians 

f. Other university employees 

g. Outside contractors 

h. Other (SPECIFY) 
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8. 



The following questions ask about where the time for project Implementation came from, or will come 
from. 



YES 



NQ 



a. Was the amount of time It has taken 
(or Is taking) to Implement your 
project substantially greater than the 
amount of time you had expected when 

you submitted your CSIP proposal? □ □ 

b. Did you or other facutty Involved In the 
project request any release time from 
regular facutty duties to do the work 

necessary to implement the project? Q lj 

c. Was any release time granted? □ □ 

d. Were any other provisions, besides 
release time (e.g., summer salary, 
hiring of technicians), made to aid 

in implementing the project? Q □ 

e. Please elaborate on "YES" responses to 8a through 8d. 



9. Have you encountered any major problems in the following areas with regard to your project? 



YE£ 



a. 



c. 



d. 



b. 



e. 



f. 



h. 



Ordering, installing and learning 

to use the equipment 

Developing course materials 

Training staff 

Integrating the equipment into the 
curriculum 

Maintaining the equipment 

Operating the equipment 

Other aspects of project implementation 

Please elaborate on "YES" answers to 9a through 9g. 



□ 
□ 
□ 



□ 
□ 
□ 
□ 



□ 
□ 
□ 



□ 
□ 
□ 
□ 
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II. RESOURCES 



The CSIP program seeks to encourage use of CSIP funds as 'seed money* for more extensive equipment 
funding efforts Your responses to questions 10 through 13 will help to illuminate the nature and extent of this 
hoped-for "multiplier effect" of CSIP grant funds. Careful attention on your part to (denotation of all ways in 
which the CSIP project may have stimulated acquisition of additional equipment and other resources is 
appreciated. NSF is also Interested In learning about the "hidden costs' of implementation of CSIP projects, 
their impact on institutions, and their policy Implications. 

Questions 10 and 11 address several kinds of multiplier effects. Question 12 asks about hidden costs 
associated with CSIP project Implementation, and Question 13 concerns the timing of availability of matching 
institutional funds. 

10. Please describe the funding sources for your project (actual or pending). 

a. How much money did NSF contribute to your 

CSIP project? $ - 

b. What are the sources and amounts of matching funds for equipment purchases? (Total 
should equal or exceed 10a above). 

1. Institution funds * 

2. Other (SPECIFY). 



Occasionally, vendors, manufacturers or others donate equipment, or provide discounts 
over and above the amount of required match. Has this happened in your project? If so 
please provide the type and approximate dollar values of each in-kind contnbution of 
equipment. (Include vendor/manufacturer upgrades; add-ons; and discounts, if above the 
standard academic discount). 

1 . "Deep" discounts (above standard discounts) $ _ 

2. Vendor upgrade or add-on $- 

3. Other sources (SPECIFY) $ — 



Sometimes It Is necessary, because of price changes, etc., to spend more than originally 
budgeted to purchase the proposed equipment. If this has been the case in your project, 
please provide the sources and dollar amounts for equipment purchases in excess of the 
required match. 

1 . Check If not applicable □ 

2. Institution funds $ — 

3. Other (SPECIFY)_ $ 



* s *2 
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Often, the 'seed* money provided by CSIP and matching funds for projects has a "multiplier 
effect," attracting further equipment or other resources to a program, beyond the original 
project. Has this happened with your project? If so, please assign a total dollar value. (Do 
not include equipment already enumerated in b, c, or d above.) 

1. Check if not applicable 

2. Additional equipment? 

3. Other additional resources? $ 

4. Please describe the nature of the expansion and indicate the source(s) of its funding 
support. 



□ 
$ 



11. As a result of your CSIP project, have you received additional space? 

□ No 

□ Yes: square feet 

12. NSF funding for CSIP projects does not cover all costs associated with project implementation. Please 
identify any "hidden costs" associated with project implementation. 

a. Facility renovation and equipment installation. 



. $ 

b. Annual cost for supplies, maintenance, and operating expenses. 

1 . Supplies and utilities $ 

2. Maintenance $ 

3. Pro-rated salaries (e.g., faculty, technicians, 

graduate students, release-time) $ 

13. How long after your CSIP grant was awarded did the matching funds become available to you? 

□ Immediately 

□ At beginning of Institution's next fiscal year 

□ Other (PLEASE EXPLAIN) 



9 
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III. IMPACTS 



!n creating the CSIP program, NSF anticipated that the projects would have an impact on faculty and on 
institutions themselves, as well as on the undergraduates who are its primary focus. Questions 14 through 22 
ask about the impacts your project has had or is expected to have in the future. NSF is particularly interested in 
any information you can provide about impacts that increase the sharing of ideas, dissemination of outcomes, 
and expansion of equipment use beyond the scope of the original project. 



14. Please assess your success to date in achieving the goals of your CSIP project. 

□ Highly successful 

□ Moderately successful 

□ Other (SPECIFY) 

1 5. Please check each of the CSIP goals listed below thai your project exemplifies. 



□ Introduction of modem instruments to improve the experiences of undergraduate students in 
science and engineering courses, laboratories and field worV 

□ Interfacing of computers with scientific instrumentation and other appropriate uses of current 
technology in science and engineering instruction 

□ Development of new ways of using instrumentation to extend instructional capabilities 

□ Establishment of equipment sharing capability via consortia or centers 



IF YOU ANSWERED. IN RESPONSE TO QUESTION 1, THAT YOUR PROJECT IS 
NOT OPERATIONAL OR IS ONLY PARTLY OPERATIONAL AT THIS TIME. PLEASE 
SKIP NOW TO QUESTION 22. OTHERWISE (IF YOUR PROJECT IS MOSTLY OR 
FULLY OPERATIONAL). CONTINUE WITH QUESTION 16. 
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16. Has your CSIP project had any of the following Impacts? 

YES NQ 

a. Improved the curriculum [j G 

b. Benefitted the faculty [I !Z 

c. Produced products that are transferrabJe to other institutlc ns [J [j 

d. Other (SPECIFY) □ □ 

e. Please elaborate on *YES" answers to 16a through I6d. 



1 7. Have you gotten any feedback on the value of your project from any of the following 9 

Y£S N£ 

a. Students 3 Hi 

b. Other faculty from your institution □ G 

c. Administration from your institution □ □ 

d. Facutty or administration from other institutions □ □ 

e. Please elaborate on "YES* answers to 1 7a through 1 76. 
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Have you noted any posith/e Impacts o( your project on students in the area of: 

YES NQ 



a. skills? □ □ 

b. conceptual or intellectual development? H □ 

c motivation or attitudes? □ □ 

d. grades? □ □ 

e. improved attendance? □ □ 

f. increased enrollments? □ □ 

g. student presentations or papers? Q □ 

h. career choices? [j Cj 

I. other? (Specify) □ □ 



j. Please elaborate on "YES" answers io 18a through 181. 



The following questions refer to the equipment purchased (or expected to be purchased) with your CSIP 
grant and matching funds for the first full year of operations or the anticipated first full year of operations, 
depending on the current status of your project. 

Estimates based on: □ Actual Full Year 

□ Anticipated Full Year 

AVERAGE 
TOTAL HOURS OF 
NUMBER USE PER 
OF PERSONS PERSON 

a. Altogether, how many faculty 

members (wil) use the equipment? 

1 . For undergraduate classroom 

or lab Instruction? 

2, For their own research? 

b. Altogether, how many undergraduates 

(will) use the equipment? 

1. In lab courses? 

2. In lecture classes? 

3. For Independent study? 

4. For student research? 



5. For other purposes? 
(SPECIFY) 



Have you or others found uses for your equipment that were not identified in your proposal or in addition 
to CSiP project activities? 



□ No 

□ Yes (ELABORATE BELOW) 



The following questions are asked to give us some indication of the extent to which results of CSIP 
projects are being shared, and in which circumstances sharing may be appropriate. 



a. Are there elements of your project that might 

interest others outside your institution 9 □ □ 

b. Have you received inquiries about your 
project from persons outside your 

institution? □ □ 

c. Have you or others shared information on 
your CSIP project in any of the following 
ways? 

1 . Presentations □ □ 

2. Publications □ □ 

3. Shared course materials □ □ 

4. Collegia! discussions □ □ 

5. Responses to inquiries □ □ 

6. Other (SPECIFY) □ □ 

d. Has Information you received about 
other CSIP projects been helpful to 

you in any way? □ □ 

e. Have any arrangements been made to share 

the equipment with other institutions? □ □ 



f . Pleas* elaborate on "YES' answers to 21 a through 21 e. 



IV. DEMOGRAPHIC AND INSTITUTIONAL INFORMATION 



To provide a context for analysis of the progress, financial, and Impact Information, NSF needs specific 
information about you, your department, your institution, and your undergraduate population. In addition, in 
order to best serve CSIP-eUgiWe institutions in the future, it will be helpful to know about your past experience 
with similar projects and proposals. 

22. Please provide the following information about yourself. 

a. What is your highest degree? 

b. In what year did you receive this degree? 

c. How many years have you been on the 

facufty of your current Institution? 

AT TIME OF 

PROPOSAL CURRENTLY 



d. What was/is your tit e? Full Professor □ □ 

Associate Professor □ □ 

Assistant Professor □ □ 

Other (SPECIFY) □ □ 



e. What was/Is your tenure status? Tenured □ □ 

Untenured □ □ 

Not Tenure Track □ [j 

f. What was/Is your teaching load, (average number of 

teaching hours per week). 



23. Please describe the facufty currently In the department where you have your primary appointment. 

NUMBER 

a. Number of facufty members 

b. Number full-time 

1 . Number with Ph.D. as highest degree 

2. Number with Masters as highest degree 

3. Number wtth Bachelors as highest degree 

c. Number part-time 
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24. Please describe the student enrollment in the department where you have your primary appointment. 

NUMBER 



a. Number of majors graduating per year 

(averaged over the last three years) 

b. Total departmental undergraduate opening 

fall enrollment In all courses in 1987 

25. Of the undergraduates affected by your project during its first full year of operation (as reported in 
Question 19b) approximately what percent were (or are expected to be): 

PERCENT 
(round to nearest 10%) 

a. female? * 

b. ethnic or racial minorities?* 

c. handicapped 7 ^ 

d. freshmen or sophomores? * 

e. upper division education majors? % 

f. upper division science** majors? ^ 

*Ethnic or racial minorities refers to students who are non-white or are of Hispanic origin. 

♦♦Math, Life Sciences, Physical Sciences, Computer Science, Engineering, Social Science, or 
Psychology. 

26. What percent of the undergraduate majors in your department will do the following after graduation. 

Check whether: □ Responses are based on records 

□ Responses are estimates, based on experience 

PERCENT 

a. go to graduate school in a related field? 

b. go to medical or other professional school? 

c. wofk at a related job In Industry or the military? 

d. work as a teacher In a related field? % 

e. other (SPECIFY! , % 
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V. THE CSIP APPLICATION PROCESS 



27. Please give your reaction to the following elements of the grant process and program, and provide your 
recommendations for those areas which need improvement. 

NEEDS 

ADEQUATE IMPROVEMENT 

a. Timing of announcements □ Q 

b. Time to respond Q Q 

c. Method of dissemination ^ 

CSIP announcements [j C 

d. Clarity of guidelines □ Q 

e. Appropriateness of guidelines □ □ 

f. Budget detail requirements □ □ 

g. Narrative detail requirements □ □ 

h. Clarity of review feedback □ □ 

I. Other (SPECIFY) □ □ 

). Please elaborate, particularly on those areas in need of improvement. 



28. How did you become actively aware of the CSIP grant program? (CHECK ALL THAT APPLY) 

□ From an NSF publication 

["] From a notice In a professional journal 

□ From a notice in a professional association newsletter, or at an association meeting 

□ From your college administration or department head 

□ Other (SPECIFY) 

29. Who wrote the grant proposal? (CHECK ONLY ONE) 

□ Yourself 

□ Department committee 

□ Other (SPECIFY) . 
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30. Who initiated the development of your CSIP grant application? (CHECK ONLY ONE) 

□ Yourself 

□ College administration 

□ Department head 

□ Other (SPECIFY) 

31 . Did you apply for funds for this or a closely related project prior to your successful application for CSIP 
funding? (CHECK ALL THAT APPLY) 

□ No (SKJP TO QUESTION 34) 

□ Yes, from CSIP, earlier year 

□ Yes, fro»n other source (SPECIFY) 

32. Did you obtain funding? 

□ No 

□ Yes (ELABORATE BELOW) 



33. If you previously applied to CSIP, did feedback provided by reviewers aid you In revising the proposal for 
resubmission? Please elaborate. 



34. In the year you received your award, were the comments from the proposal reviewers useful? 

□ No 

□ Yes (SPECIFY)^ ■ 

35. Prior to submitting the proposal for your CSIP project had you successfully obtain^ grant funding for 
any other project(s)? (CHECK ALL THAT APPLY) 

□ No 

□ Yes, Federal source 

□ Yes, other source 
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How long did It take to HI out this form? 

Minutes 

Please provide your namo and phone number, in case additional information or clarification is needed. 

Name: . 



Phc* Number: / 



eric 
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OMB #3145-01 13 
Expires 4/W 



ASSESSMENT OF THE COLLEGE SCIENCE INSTRUMENTATION PROGRAM 

APPLICANT QUESTIONNAIRE 

National Scienct Foundation 



The National Science Foundation Is conducting an assessment of the College Science Instrumentation 
Program (CSIP). The purpose of this survey is to determine the impact of CSIP as well as to find ways that 
the grant program might be improved. To that end we are surveying both grantees and applicants whose 
proposals were not funded by CSIP. 

We ask that the requested information be obtained from the proposed principal investigator whose name 
and address, and proposed CSIP project title and year of proposal, appear below. Piease correct the label 
If any of the information is incorrect. 



Where exact cost (or other) data are not available, estimates are acceptable, 
than ours.) 



(Your estimates will be better 



All information you provide is confidential and will be published only in aggregate form. Your response, 
though important for an accurate assessment, is voluntary and failure to provide some or all of the 
information will in no way affect you or your institution. This Information is solicited under the authority of 
the National Science Foundation Act of 1950, as amended. 

PImm rttum this form by D •camber 22. Your cooperation in returning the survey questionnaire 
promptly Is essentia] to the timely completion of the assessment. Please return the completed survey to: 

WESTAT, INC. 
1650 Research Blvd 
RockvHIe, MD 20850 



If you have any questions regarding this survey, please contact Ms. Carin Ceiebuski at Westat's toll-free 
number 800/937^8281 , or contact Mr. David Rorio of NSF at 202/357-7425. 



Public reporting burden for this cdlecfcun of information it estimated to average 15 minutes per response, including the time for 
reviewing instructions, searching existing data sources, gathshng and maintaining ttts data needed, and complttir.g and reviewing 
the collection of information. Send comments rsga/ding this burden sttimats or any other aspect of this collection of information, 
including suggestions for reducing this burden, to: 



Herman G. Fleming 
Clearance Officer • Room 206 
Division of Personnel and Management 
National Science Foundation 
Washington, D.C, 20550 



and to 



Office of Management and Budget 
Paperwork and Rsduction Project (3145-0113) 
Washington, D.C., 20503 
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I. THE CSIP APPLICATION PROCESS 



NSF Is interested in learning as much as possible about reactions to the CSIP application process, the 
prior funding experience of applicants, and the nature of the proposal process, as well as in hearing any specific 
recommendations which might improve the CSIP program. Respondents are urged to use as many extra pages 
as necessary to elaborate fully on aspects of the proposal process which are in need of improvement. NSF will 
use this information to examine how its programs might be made more responsive to the needs of institutions 
like yours. 

Later sections of this gyestionrwe deal with outcomes of your CSIP application and with demographic 
information to be used for contextual analysis of CSIP study data. 

1 . How did you become actively aware of the CSIP grant program? (CHECK ALL THAT APPLY) 

□ From an NSF publication 

□ From a notice in a professional journal 

□ From a notice in a professional association newsletter, or at an association meeting 

□ From your college administration or department head 

□ Other (SPECIFY) 

2. Who wrote the grant proposal? (CHECK ONLY ONE) 

□ Yoursetf 

□ Department committee 

□ Other (SPECIFY) 

3. Who initiated the development of your CSIP grant application? (CHECK ONLY ONE) 

□ Yoursetf 

□ Cdlege administration 

□ Department head 

□ Other ^SPECIFY) . 

4. Did you apply for funds for this or a closely related project prior to your application for CSM funding? 

□ No (SKIP TO QUESTION 6) 

□ Yos (SPECIFY SOURCE) 

5. Did you obtain any funding at that time? 

□ No 

□ Yes (ELABORATE BELOW) 
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6. Prior to submitting the CSIP proposal had you successfully obtained grant funding for any other 
project(s)? (CHECK ALL THAT APPLY) 

□ NO 

□ Yes, Federal source 

□ Yes, other source 

7. Please check each of the CSIP goals listed below that your proposed project exemplified. 

□ Introduction of modem instruments to improve the experiences of undergraduate students in 
science and engineering courses, laboratories and field work 

□ Interlacing of computers with scientific instrumentation and other appropriate uses of current 
technology in science and engineering instruction 

□ Development of new ways of using instrumentation to extend instructional capabilities 

□ Establishment of equipment sharing capability via consortia or centers 

8. Did you find the CSIP reviewer's comments on your proposal useful in any way? 

□ No 

□ Yes (EXPLAIN HOW) 



Please give your reaction to the following elements of the grant process and program, and provide your 
recommendations on those areas which need improvement. 

NEEDS 

ADEQUATE IMPROVEMENT 

a. Timing of announcements □ Q 

b. Time to respond □ Q 

c. Method of dissemination of 

CSIP announcements □ Q 

d. Clarity of guidelines □ □ 

e. Appropriateness of guidelines □ Q 

f. Budget detal requirements □ □ 

g. Narrative detail requirements □ □ 

h. Clarity of review feedback □ □ 

I. Other (SPECIFY) □ □ 

I. Please elaborate, particularly on those areas in need of improvement 
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II. OUTCOMES OF YOUR CSIP APPLICATION 



NSF is interested in whether unsuccessful CSIP applicants pursued other funding sources (either 
institutional or external) for their equipment, whether they were successful in funding rt, and whether the CSIP 
application process Itself had any posith/e or negative effects on the school, the department, or individuals. NSF 
believes that Increased understanding of these outcomes and effects will enable it to improve policies and 
procedures and urges careful consideration of responses and complete answers to open-ended questions. If 
additional space is needed, please use the space provided in question 20. 



10. Since your CSIP grant application, have any other efforts been made (by you or others) to obtain funding 
support for ycur proposed project? 

□ No (SKIP TO QUESTION 1 2) 

□ Yes (PLEASE DESCRIBE) 



1 1 . Has any funding support been obtained for your proposed project? 

□ No 

□ Yes (PLEASE DESCRIBE) 



12. Have you been able to obtain any of the instruments (or equivalents) that you :equested in your CSIP 
application? 

□ No (SKIP TO QUESTION 15) 

□ Yes 

1 3. Approximately what percent of your original CSIP budget request do the obtained instruments represent? 
Percent 

(If percent Is unknown, please Indicate the combined purchase price of the obtained 
instruments: $ _) 

1 4. What funding sources contributed to the acquisition of these instruments. (CHECK ALL THAT APPLY) 

□ Department funds 

□ Other college/university funds 

□ Federal sources (SPECIFY) 

□ Business/Industry 

□ Other (SPECIFY! 
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15. 



Are any further efforts to obtain funding support for this project anticipated? 

□ No 

□ Yes (PLEASE DESCRIBE) 



1 6. Did the development of your CSIP proposal have any direct or indirect positive effects? 

□ No 

□ Yes (PLEASE ELABORATE) 



1 7. Did the unsuccessful outcome of your CSIP grant application have any direct or indirect negative effects 
on the school, the department or on Individual students or faculty? 

□ No 

□ Yes (PLEASE ELABORATE) 
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III. DEMOGRAPHIC AND INSTITUTIONAL INFORMATION 



To provide a context for analysis of information from sections I and II and for comparative analysis as 
part of the overall CSIP study, NSF needs some information about you, your department, your institution, and 
your undergraduate population. 

1 8. Please provide the following information about yourself. 

a. What is your highest degree? 

b. In what year did you receive this degree? 

c. How many years have you been on the 

faculty of your current institution? 

AT TIME OF 

PROPOSAL CURRENTLY 



d. What was/Is your title? Full Professor □ □ 

Associate Professor □ □ 

Assistant Professor □ Q 

Other (SPECIFY) □ Q 



e. What was/is your tenure status? Tenured □ □ 

Untenured □ □ 

Not Tenure Track □ □ 



f. What was/Is your teaching load, (average number 

of teaching hours per week). 

19. Please describe the faculty currently in the department where you have your primary appointment. 

NUMBER 

a. Number of faculty members 

b. Number full-time 

1 . Number wtth Ph.D. as highest degree 

2. Number wtth Masters as highest degree 

3. Number with Bachelors as highest degree 

c. Number part-time 



^8 



20. Please describe the student enrollment In the department where you have your primary appointment. 

NUMBER 

a. Number of majors graduating per year 

(averaged over the last three years) 

b. Total departmental undergraduate opening 

fall enrollment In 1987 

21. Use this space to write in any additional comments about CSiP or for continuations of answers to 
previous questions. 



How long did it take to fill out this form? 

Minutes 

Please provide your name and phone number, in case additional information or clarification are needed. 

Name: 



Phone Number: / 
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CSIP - Site Visit Summary Report 



Date of Visit ~ 

Project Number 

School 

Principal Investigator 
Department Chairman 

Dean 

Westat Staff 

Consultant 



Use this form to summarize information obtained about each project by 
Westat staff and consultants during site visit interviews and other 
activities. Both members of the site visit team will meet after the site visit 
to discuss the questions and agree on responses. 



V*0 



1. Are there any updates/changes in project status as reported in the questionnaire? 



□ Same as in questionnaire 

□ Project is further along than presented in questionnaire 

□ Project is less far along than presented in questionnaire 
Z2 Not ascertained 

□ Not applicable 

What kind of impact has the project had on: 



POSITIVE 


NO 


NEGATIVE 


NOT 


NOT 




IMPACT 


IMPACT 


IMPACT 


ASCERT. 


APPLIC. 


Curriculum and instruction - 












new/modified courses 


□ 


□ 


□ 


n 


. — | 


new/upgraded topics 


□ 


□ 


□ 


□ 




teaching of subject 


□ 


□ 


□ 


□ 




Department faculty - 












morale 


□ 


□ 


□ 


□ 


— 


LQlCrCbl III UpgTdUJiig 


l — i 

u 


□ 


□ 


□ 


— ! 


recruitment 


□ 


| \ 

□ 


□ 


□ 


i i 


Principal Investigator - 












Dossibilitv for fpmirp 


n 
i — i 


□ 


□ 


□ 


r— 1 
i i 


salary 


□ 


I I 

u 


i — i 
u 


i — i 
U 




work load 


□ 


□ 


□ 


□ 


□ 


other professional obligations 


□ 


□ 


□ 


□ 




Students - 












level of interest 


□ 


□ 


□ 


□ 




understanding of subject 


□ 


□ 


□ 


□ 




preparation 


□ 


□ 


□ 


□ 


lJ 


Science instruction - 












usefulness as model 


□ 


□ 


□ 


□ 


□ 


Funding - 












acquiring matching funds 


□ 


□ 


□ 


□ 


□ 


leveraging additional money 


□ 


□ 


□ 


□ 


□ 


Department - 












prestige 


□ 


□ 


□ 


□ 


□ 


funding level 


□ 


□ 


□ 


□ 


□ 


space 


□ 


□ 


□ 


□ 


□ 


student enrollment 


□ 


□ 


■ □ 


□ 


□ 


student recruitment 


□ 


□ 


□ 


□ 


□ 


faculty recruitment 


□ 


□ 


□ 


□ 


□ 


instructional program 


□ 


□ 


□ 


□ 


□ 


Institution - 












grant administration proce ss 


□ 


□ 


□ 


□ 


□ 


institutional support 


□ 


□ 


□ 


□ 


□ 


prestige 


□ 


□ 


□ 


□ 


□ 


student recruitment 


□ 


□ 


□ 


□ 


□ 
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In what areas has the institution given support to the project: 





aUMt 










SUPPORT 


SUPPORT 


PROBLEM 


NOT 


- 


GIVEN 


GIVEN 


AREA 


ASCERT. 


At the proposal stage - 










initiation of proposal 


□ 


□ 


□ 


□ 


encouragement of submission 


□ 


□ 


□ 


G 


preparation of proposal 


□ 


□ 


□ 


□ 


At the implementation stage - 










release time 


□ 


□ 


□ 


□ 


additional space 


□ 


□ 


□ 


□ 


supplies 


□ 


□ 


□ 


□ 


technical support 


□ 


□ 


□ 


! — 1 


provisions for 










maintenance and repair 


□ 


□ 


□ 


□ 



What is the climate for innovation and change in the department and the institution: 



YES 

a. Innovation and change are seen as desirable - 

in the department □ 
in the institution □ 

b. Program planning, development, and 
evaluation are done on a formal basis - 

in the department □ 
in the institution □ 

c. The project was part of planned change - 

in the department □ 
in the institution □ 

d. The P.I.'s efforts with respect to the 
project are recognized as valuable - 

by the department □ 
by the institution □ 

e. Is the Department Chairman well 
acquainted with the project? 

(if too new check not applicable) □ 

f. Is the Dean well acquainted with 
the project? (if too new check 

not applicable) □ 



NO 

□ 
□ 



□ 
□ 



□ 
□ 



n 
□ 



NOT 
ASCERT. 



NOT 
APPLIC. 



□ 
□ 



□ 
□ 



□ 
□ 



□ 
□ 



□ 



□ 



□ 
□ 



□ 
□ 



□ 
□ 



□ 
□ 



□ 




Rate the success of this CSIP project in terms of: 



a. Achievingnts original goals 

b. Introducing modern instruments to 
improve the experiences of under- 
graduate students in laboratories 
and field work 

c. Interfacing computers with scientific 
instrumentation for instruction 
purposes 

d. Developing new ways of using instru- 
mentation to extend instructional 
capabilities 

e. Establishing equipment sharing • 
capability via consortia or centers 

f. Leveraging additional funds 
(beyond the required match) 



SOT NOT 
HIGH MODERATE NONE ASCERT. APPLIC. 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



□ 



L „ 



□ 



Have any of the following areas caused problems which held up project implementation or required the P.I. to 
expend more effort than anticipated to keep the project on schedule: 



a. Acquiring funds 

b. Ordering, installing, and 
learning to use equipment 



MAJOR 

□ 

□ 



Operating and/or maintaining equipment □ 



NOT NOT 
MINOR NONE ASCERT. APPLIC. 



□ □ □ 

□ □ □ 

□ □ □ 



□ 
□ 



What would have happened to this project in the absence of CSIP funds: 

□ Would have been mostly or fully implemented with institution funds 

□ Would have been mostly or fully implemented with outside funds 

□ Would have been mostly or fully implemented with a mix of funds 

□ A scaled-down version would have been implemented 

□ Would not have been implemented 

□ Not ascertained 

□ Not applicable 
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Did the questionnaire responses contain any significant omissions or exaggerations regarding: 



QUEX. QUEX. 

UNDER OVER- QUEX. NOT NOT 

STATES STATES ACCURATE AS CERT. APPLIC. 



a. 


Extent of progress In 
implementing project 


□ 


□ 


□ 


□ 




b. 


Extent of changes from 
planned equipment 


□ 


□ 


□ 


□ 


i — j 

. — ! 


c. 


Extent of changes from 
planned objectives 


□ 


□ 


□ 


□ 


1 — I 


d. 


Extent of resource leveraging 


□ 


□ 


□ 


□ 




e. 


Extent of other project impacts 


□ 


□ 


□ 


□ 


□ 


f. ' 


Type and extent of dissemination 


□ 


□ 


□ 


□ 


□ 



What role did each of the following play in the success of the project in achieving CSIP objectives? 

BOTH/ 

POSITIVE /BENEFICIAL NEITHER/ NEGATIVE/ 







HIGHLY 




SLIGHTLY 


SJi. 


IMPEDIMENT 






3 


2 


1 


0 


-1 


a. 


P.I.'s initiative, effort, 


♦ 












time-on-task 


□ 


□ 


□ 


□ 


□ 


b. 


Other faculty 


□ 


□ 


□ 


□ 


□ 


c. 


Department administrators 


□ 


□ 


□ 


□ 


□ 


d. 


Institution administrators 


□ 


□ 


□ 


□ 


□ 


e. 


Student Assistants 


□ 


□ 


□ 


□ 


□ 


f. 


Others (specify) 


□ 


□ 


□ 


□ 


□ 



Report briefly on questionnaire feedback from the respondent. 
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11. Report briefly on other recommendation from P.I. concerning CSIP, NSF, and dissemination. 



12. How well is this project doing, and why? 
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NATIONAL SCIENCE FOUNDATION 

WASHINGTON, D.C. 20550 



OFFICIAL BUSINESS 
PENALTY FOR PRIVATE USE $300 



RETURN THIS COVER SHEET TO ROOM 233. IF YOU DO 
N07 WISH TO RECEIVE THIS MATERIAL D , OR IF 
CHANGE OF ADDRESS IS NEEDED 2 , INDICATE 
CHANGE, INCLUDING ZIP CODE ON THE LABEL (DO 
NOT REMOVE LABEL). 



